{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Python|分分钟教你学会写KNN算法 "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 1. 什么是KNN算法?"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**K最近邻**(k-Nearest Neighbor，KNN)分类算法是一个比较成熟也是最简单的**机器学习**(Machine Learning)算法之一。该方法的思路是：如果一个样本在**特征空间**中与K个实例最为相似(即特征空间中最邻近)，那么这K个实例中大多数属于哪个类别，则该样本也属于这个类别。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "其中，计算样本与其他实例的相似性一般采用**距离衡量法**。离得越近越相似，离得越远越不相似。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "![KNN示意图](img/../images/1.jpg)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "如上图所示，K=3，距离绿色样本最近的3个实例中（圆圈内），有两个是红色三角形（正类）、一个是蓝色正方形（负类）。则该样本属于红色三角形（正类）。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 2. KNN算法本质"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "我们知道,一般机器学习算法包括两个过程：**训练过程**和**测试过程**。训练过程通过使用机器学习算法在训练样本上迭代训练，得到较好的机器学习模型；测试过程是使用测试数据来验证模型的好坏，通过正确率来呈现。KNN算法的本质是在训练过程中，它将所有训练样本的输入和输出标签(label)都存储起来。测试过程中，计算测试样本与每个训练样本的距离，选取与测试样本距离最近的前k个训练样本。然后对着k个训练样本的label进行投票，票数最多的那一类别即为测试样本所归类。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "其实，KNN算法非常简单，可以说在训练过程中基本没有算法参与，只有**存储训练样本**。可以说KNN算法实际上是一种**识记类算法**。因此，KNN虽然简单，但是其明显包含了以下几个缺点：\n",
    "\n",
    "* 整个训练过程需要将所有的训练样本极其输出label存储起来，因此，**空间成本**很大。\n",
    "\n",
    "* 测试过程中，每个测试样本都需要与所有的训练样本进行比较，运行**时间成本**很大。\n",
    "\n",
    "* 采用距离比较的方式，分类**准确率**不高。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "好了，介绍完了KNN算法的理论知识之后，我相信大家都跃跃欲试了。接下来，我们就来手把手教大家使用Python实现一个KNN分类问题，进入机器学习实战大门。开始吧～"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 3. 数据准备"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 数据下载与归类"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "首先，数据集我们选择经典的鸢尾花卉数据集（Iris）。Iris数据集每个样本x包含了花萼长度（sepal length）、花萼宽度（sepal width）、花瓣长度（petal length）、花瓣宽度（petal width）四个特征。样本标签y共有三类，分别是Setosa，Versicolor和Virginica。Iris数据集总共包含150个样本，每个类别由50个样本，整体构成一个150行5列的二维表，如下图展示了10个样本："
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "![](img/../images/2.png)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "如何获取这些数据呢？很简单，我们可以使用代码，直接从网上下载，下载后的数据集存放在'../data/'目录下。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 27,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "import numpy as np\n",
    "import pandas as pd\n",
    "\n",
    "#data = pd.read_csv('https://archive.ics.uci.edu/ml/machine-learning-databases/iris/iris.data', header=None)    # 下载iris数据集\n",
    "data = pd.read_csv('./data/iris.data.csv', header=None)\n",
    "data.columns = ['sepal length', 'sepal width', 'petal length', 'petal width', 'species']    # 特征及类别名称"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "然后，我们将三个类别的数据分别提取出来，setosa、versicolor、virginica分别用0、1、2来表示。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 28,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "X = data.iloc[0:150, 0:4].values\n",
    "y = data.iloc[0:150, 4].values\n",
    "y[y == 'Iris-setosa'] = 0                                 # Iris-setosa 输出label用0表示\n",
    "y[y == 'Iris-versicolor'] = 1                             # Iris-versicolor 输出label用1表示\n",
    "y[y == 'Iris-virginica'] = 2                              # Iris-virginica 输出label用2表示\n",
    "X_setosa, y_setosa = X[0:50], y[0:50]                     # Iris-setosa 4个特征\n",
    "X_versicolor, y_versicolor = X[50:100], y[50:100]         # Iris-versicolor 4个特征\n",
    "X_virginica, y_virginica = X[100:150], y[100:150]         # Iris-virginica 4个特征"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "接下来看一下三种类别不同特征的空间分布。为了可视性，我们只选择sepal length和petal length两个特征，在二维平面上作图。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 29,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXwAAAEKCAYAAAARnO4WAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMS4wLCBo\ndHRwOi8vbWF0cGxvdGxpYi5vcmcvpW3flQAAIABJREFUeJzt3XmcVPWZ7/HPQwNCCyRucaGHRWfc\nQRFUXK4a45JRx+WKL0kwKuoQ4Ep0cp2ocYwkaLwzmhuTuW6YiBtRDK4xk0RxRKNx1G5tNg3qGNQG\nlAaVgLLz3D9OFXQ31VWnus45tZzv+/U6r+4661NHfOpXT/9+v2PujoiI1L5u5Q5ARESSoYQvIpIS\nSvgiIimhhC8ikhJK+CIiKaGELyKSEkr4IiIpoYQvIpISSvgiIinRvdwBtLXzzjv7oEGDyh2GiEjV\naGpqWu7uu4TZt6IS/qBBg2hsbCx3GCIiVcPM3g+7r0o6IiIpoYQvIpISSvgiIikRWw3fzPYBZrRZ\ntSfwA3e/pZjzbNiwgZaWFtauXRtpfGnUq1cvGhoa6NGjR7lDEZEyiC3hu/tC4GAAM6sDFgOPFXue\nlpYW+vbty6BBgzCziKNMD3dnxYoVtLS0MHjw4HKHIyJlkFRJ52vAf7t76L8mZ61du5addtpJyb5E\nZsZOO+2kb0oiKZZUwh8NPNjVg5Xso6H7KJJusffDN7OewOnA1Z1sHweMAxgwYEDc4YiIdFm/G/ux\nav2qbdb37dmXv1791zJEVJwkWvh/D7zu7h/n2ujuU919hLuP2GWXUIPFKtY999zDkiVLyh2GiMQk\nV7LPt77SJJHwv0EJ5ZxqooQvIpUs1oRvZvXAicCjcV6nnenTYdAg6NYt+Dl9ekmn+/zzzzn11FM5\n6KCDOPDAA5kxYwZNTU0ce+yxDB8+nJNPPpmlS5cyc+ZMGhsbGTNmDAcffDBr1qzh2WefZdiwYQwZ\nMoSLLrqIdevWAXDVVVex//77M3ToUK644goAfvOb33D44YczbNgwTjjhBD7+OOcXIhGRrnP3ilmG\nDx/uHb355pvbrOvUAw+419e7w9alvj5Y30UzZ870Sy65ZMvrzz77zI844ghftmyZu7s/9NBDPnbs\nWHd3P/bYY/21115zd/c1a9Z4Q0ODL1y40N3dv/Wtb/lPf/pTX7Fihe+9996+efNmd3f/9NNP3d39\nk08+2bLurrvu8u9+97tdjjmfou6niLTDZDpdyhYTNHrIHFtbI22vuQa++KL9ui++CNZ30ZAhQ5g1\naxZXXnklf/zjH/nwww+ZP38+J554IgcffDDXX389LS0t2xy3cOFCBg8ezN577w3ABRdcwAsvvEC/\nfv3o1asXl1xyCY8++ij19fVAMN7g5JNPZsiQIdx0000sWLCgyzGLiORSWwn/gw+KWx/C3nvvTVNT\nE0OGDOHqq6/mkUce4YADDqC5uZnm5mbmzZvH008/vc1xwQfvtrp3786rr77K2WefzeOPP87Xv/51\nACZNmsSll17KvHnzuPPOO9VfXqQC9e3Zt6j1laa2En5n3TpL6O65ZMkS6uvrOe+887jiiit45ZVX\naG1t5eWXXwaCqR+yrfG+ffuyalXw1/p9992XRYsW8e677wJw//33c+yxx7J69WpWrlzJKaecwi23\n3EJzczMAK1eupH///gDce++9XY5XROJT7b10Kmo+/JLdcAOMG9e+rFNfH6zvonnz5vHP//zPdOvW\njR49enD77bfTvXt3vvOd77By5Uo2btzI5ZdfzgEHHMCFF17I+PHj6d27Ny+//DLTpk3jnHPOYePG\njRx66KGMHz+eTz75hDPOOIO1a9fi7vz0pz8FYPLkyZxzzjn079+fkSNH8pe//KXUuyEi0o51Vnoo\nhxEjRnjHB6C89dZb7LfffuFPMn16ULP/4IOgZX/DDTBmTMSRVq+i76eIbGE/7Hy0ul9XnlxqZk3u\nPiLMvrXVwocguSvBi0gXVPtI2kJqq4YvIlKCaq/RF6KELyKSEkr4IiIpUXs1fBGRmJTrD7NRUQtf\nRCQllPDL4Ac/+AGzZs0q+rjZs2dz2mmnxRCRiED1j6QtRCWdmGyZrKjbtp+pP/rRjxKJYePGjXTv\nrv/EImEl0fWynF0/a7KFv3Qp7LUXfPRR6ee68sorue2227a8njx5Mj/5yU+46aabOPTQQxk6dCjX\nXXcdAIsWLWK//fZj4sSJHHLIIXz44YdceOGFHHjggQwZMmTLqNoLL7yQmTNnAvDaa69x5JFHctBB\nB3HYYYexatUq1q5dy9ixYxkyZAjDhg3jueee2yauTz75hDPPPJOhQ4cycuRI5s6duyW+cePGcdJJ\nJ3H++eeXfgNEJFLl7PpZkwl/yhRYtCj4WarRo0czY8aMLa8ffvhhdtllF9555x1effVVmpubaWpq\n4oUXXgCCWTLPP/983njjDZYvX87ixYuZP38+8+bNY+zYse3OvX79es4991x+9rOfMWfOHGbNmkXv\n3r259dZbgWBahwcffJALLrhgm8nUrrvuOoYNG8bcuXP58Y9/3C65NzU18cQTT/CrX/2q9BsgIjWj\n5r7vL10K06bB5s3Bz2uvhd126/r5hg0bxrJly1iyZAmtra3ssMMOzJ07l6effpphw4YBsHr1at55\n5x0GDBjAwIEDGTlyJAB77rkn7733HpMmTeLUU0/lpJNOanfuhQsXsvvuu3PooYcC0K9fPwBefPFF\nJk2aBASTsA0cOJC333673bEvvvgijzzyCADHH388K1asYOXKlQCcfvrp9O7du+tvWqQC1foo2CTU\nXAt/ypQg2QNs2hRNK3/UqFHMnDmTGTNmMHr0aNydq6++essUye+++y4XX3wxANtvv/2W43bYYQfm\nzJnDcccdx6233soll1zS7rzujtm2c3OEmd8o1z7Zc7WNQaRW1Poo2CTUVMLPtu7Xrw9er18fvC61\nlj969GgeeughZs6cyahRozj55JO5++67Wb16NQCLFy9m2bJl2xy3fPlyNm/ezNlnn82UKVN4/fXX\n223fd999WbJkCa+99hoAq1atYuPGjRxzzDFMzzya8e233+aDDz5gn332aXds231mz57NzjvvvOUb\ngohILjVV0mnbus/KtvIzZfEuOeCAA1i1ahX9+/dn9913Z/fdd+ett97iiCOOAKBPnz488MAD1NXV\ntTtu8eLFjB07ls2ZoG688cZ223v27MmMGTOYNGkSa9asoXfv3syaNYuJEycyfvx4hgwZQvfu3bnn\nnnvYbrvt2h07efJkxo4dy9ChQ6mvr9cc+iJVom/Pvp2WpuJWU9MjNzTA4sXbru/fH3I8hTCVND2y\nVKtKnJq4EqR2emQldRGRztVUDV9Ealetj4JNQk218EWkOoXpclmo62US3TarvWuoWvgiUnZRdLlM\nottmtXcNjTXhm9mXzWymmf3ZzN4ysyPivJ6IiHQu7pLOz4Dfu/soM+sJ1Md8PZGKUe1f/6X2xNbC\nN7N+wDHALwHcfb27fxbX9ZK0ZMkSRo0aVfRxl1xyCW+++Wbefe644w7uu+++roYmFaTav/5L7Ymz\nhb8n0ApMM7ODgCbgMnf/PMZrJmKPPfbYMttlW4WmI/7FL35R8Nzjx48vKTYRkc7EmfC7A4cAk9z9\nFTP7GXAVcG3bncxsHDAOYMCAASVdMI6v0FdeeSUDBw5k4sSJQDDCtW/fvkybNo358+dzzz338Nvf\n/pa1a9fy+eefM2vWLC699FKef/55Bg8ezObNm7nooosYNWoUxx13HDfffDMjRoygT58+XHbZZTz1\n1FP07t2bJ554gl133ZXJkyfTp08frrjiCt59913Gjx9Pa2srdXV1/PrXv2bXXXfljDPO4NNPP2XD\nhg1cf/31nHHGGSXdN0m3WundksQI1nKOko1CnAm/BWhx91cyr2cSJPx23H0qMBWCkbalXDCOr9Cj\nR4/m8ssv35LwH374Ye644w6mTZu2ZZ+XX36ZuXPnsuOOOzJz5kwWLVrEvHnzWLZsGfvttx8XXXTR\nNuf9/PPPGTlyJDfccAPf+973uOuuu/iXf/mXdvuMGTOGq666irPOOou1a9eyefNmevbsyWOPPUa/\nfv1Yvnw5I0eO5PTTT885CZtIGLXSuyWJv4tU+99eYkv47v6RmX1oZvu4+0Lga0D+AnYFyjU9csdv\nIieeeCI77rgjEExbfM4559CtWzd22203vvrVr+Y8b8+ePbc8rnD48OE888wz7bavWrWKxYsXc9ZZ\nZwHQq1cvADZs2MD3v/99XnjhBbp168bixYv5+OOP2a2UOaBFJBXi7qUzCZie6aHzHjC2wP4VKTs9\n8kcffcTo0aO32d52OuKwcxP16NFjS6u8rq6OjRs3ttve2XmmT59Oa2srTU1N9OjRg0GDBm3zcBSp\nDNX+9T9JulfJiDXhu3szEGpSn0o2evRo/vEf/5Hly5fz/PPPs27duk73Pfroo7n33nu54IILaG1t\nZfbs2Xzzm98s+pr9+vWjoaGBxx9/nDPPPJN169axadMmVq5cyVe+8hV69OjBc889x/vvv1/KW5MY\nRfH1v9Tat7qGSlsaaRtCx+mR8zn77LNpaGjgwAMP5Nvf/jaHH344X/rSl7p03fvvv5+f//znDB06\nlCOPPJKPPvqIMWPG0NjYyIgRI5g+fTr77rtvl84t1aHU2ne1dA2tljirXU1Nj1wprZnVq1fTp08f\nVqxYwWGHHcZLL71UMTV2TY9cXUqdEjjM8ZXQS0dTH3ddaqdHrpSvqKeddhqfffYZ69ev59prr62Y\nZC+SSxKTklXK/5tpV1MJv1LMnj273CGIREblltpRFTX8Sio7VTPdR5F0q/gWfq9evVixYgU77bST\nBheVwN1ZsWLFlv78kl8UZYxCdekwdeu0dFdMy/sst4pP+A0NDbS0tNDa2lruUKper169aGhoKHcY\nVaFSyhhpqX2n5X2WW8Un/B49ejB48OByhyEiUvWqooYvIuWjZ8nWjopv4YukWSX0kVe5pXaohS9S\nwWplJkupDEr4IjnUShmjVt6HREMlHZEcopiYrBKmBKiECdykcqiFL9JFaSmFpOV9poESvohISijh\ni1SwJGrwqvOnh2r4IhVMz2mVKKmFLyKSEkr4Il2UllJIWt5nGqikI9JFSZRCKuGZtir51A618EUq\nWFqeaSvJUMIXEUkJlXSk6hR6cEgxZYylS+Hoo+GllyDqRw9Xw0jcMDTStnaohS81p5gyxpQpsGhR\n8LOccVSyWnkfEnPCN7NFZjbPzJrNrDHOa4kUa+lSmDYNNm8Ofn70UbkjEolXEiWdr7r78gSuIzUi\n0TngrwperwF2v7P2yhR6Vqy0pZKOVBzNAR+d7N8KJnzsdPuRM3GZ49d5TX2oSXhxJ3wHnjazJjMb\nF/O1RCQHla4kK+6SzlHuvsTMvgI8Y2Z/dvcX2u6Q+SAYBzBgwICYw5FyS6JcE0UZo1J6pkTxXqZM\nCZI9wKZNwetbb002BqkMsSZ8d1+S+bnMzB4DDgNe6LDPVGAqwIgRI6qjn5p0WRSllELdGaNIyJVS\n8in1vWRb9+vXB6/Xrw9eX3tt+G6o2RgmToQ774Tx44v7wJDKEVtJx8y2N7O+2d+Bk4D5cV1PRLbV\ntnWflW3lF0NlodoQZw1/V+BFM5sDvAr81t1/H+P1pEZoDvjoPPnk1tZ91vr18MQTxZ0nV1lIqk9s\nJR13fw84KK7zS+2KqkYe5yhaqJw6fz4tLeH37ex+RVEWksqgbplSs/KNoo2iRl8pdf6odHa/oioL\nSfkVTPhmtp2ZfdPMvm9mP8guSQQntSepUkqpNeco4rT1ufftbH055btfUZWFpPzClHSeAFYCTcC6\neMORWpdUqaPUrojZOJub4dBDoakJhg4tLobNNwTniLu0FEahGPLdryjKQlIZwpR0Gtz9XHf/N3f/\nSXaJPTKRLuqs5tyVniXnnQcbN8I3v9n1eOKcoC2KGKK8X5XwXqVzYRL+n8xsSOyRiEQkqppzczMs\nWBD8vmABzJ1bfCyV0J2xUAzqupkenSb8zCyXc4GjgdfNbKGZzW2zXqQihak5h6nRn3de+20dW/lh\nzhFVd8alS2GvvTpPovm2F4pBXTdTxN1zLsDAfEtnx5WyDB8+3EUqwRtvuMO2y5w54c+xZIl7r17t\nj+/d233p0uLjmTDBvVs394kTi9seZQz5JHUd2RbQ6CFzbKctfHd/393fB67P/t52XdwfRCLl1LF1\nn1VMLT+pUkm+7Ul1qVTXzeoQpoZ/QNsXZlYHDI8nHJHK8NZbxa3PJalSSb7tSXWpVNfN6pCvhn+1\nma0ChprZXzPLKmAZQVdNiVC/G/thP7Rtln439it3aDkVqilHcXyp1wijuRl69Nj2D7ItLdCrV/t1\nvXvD4sXh42xpyVUU2rabY773WagHTaHtYWMoVVLXkdLkK+nc6O59gZvcvV9m6evuO7n71QnGmArV\nNmqz1O53YY5PootfZ90uiylRxHkvCsWhUooUw4Kaf54dzA7JsXol8L67b4wymBEjRnhjYzoffWs/\ntE63FZoOOGlLl8Kee8LatUGr9733ihtkE+b4Uq8RRnMzDBu29fWcOVsHVzU05G7N9+/fvtUa970o\nFEfYOKV2mVmTu48Is2+YGv5twH8RzFl/V+b3h4C3zeykLkcpVStM97tSugmG3adU+bpdhi1RxH0v\nCsWR3b5kSfDBsXRp8WUjSZFC3XgIkvsBbV7vD0wD9gSaw3YHCrOkuVsmk+l0qSRhu9+V0k0wiS5+\nSXa7TKLLZFe7bUr1I4pumW3s6+4L2nxAvAkM82D6Y0mZMDXjUrsJJlGXTqrbZRJdJkvptinpEibh\nLzSz283s2MxyG0E5ZztgQ8zxpUa1PJAjTPe7UrsJJtHFL6lul0l0mSyl26akTKGvAEBv4H8DjwGP\nA1cA9QQfFn3CfpUIs6S5pFMr0jjistSSTSnllkLXSON/j7QhypKOu6/xYIbMs9z9THe/2d2/cPfN\n7r46vo8iqUZp6yZYasmm1HKLum1KMcI8AOUoM3vGzN42s/eySxLBSfVJ24jLUks2pZZbCl0jbf89\nJL8w/fD/DPwTwQNQNmXXu/uKqINJcz/8tAnzoIxSHj4S9hqlaNuHPquYvvilHi8C0ffDX+nuv3P3\nZe6+IruUGKOkXJjRqaU+fCTukbqllktUbpGkhWnh/x+gDniUNo84dPfXow5GLfx0CDM6Nd8o2Kiu\nUapSR7lqlKxEIeoW/uHACODHwE8yy81dD0/SLkzdutDDR6D00bylKnXCME04Jkkr2MJPklr4tS9M\n3bpj6z6rYyt/4kS4804YP779A8pVG5c0ibSFb2a7mtkvzex3mdf7m9nFpQYp6RSmbh1mFGwlPPRD\npNqEKencA/wB2CPz+m3g8rAXMLM6M3vDzJ4qPjypVp3NMx+mm2CYUbBTpgRJHII/7HZ1BGspz4oV\nqTZhEv7O7v4wsBnAgymRN+U/pJ3LgCIGrEst6KyHzWuv5X6wSNtKXqGHj2Rb9xsyE3ts2ND1h34U\n6smTxJz8IkkJk/A/N7OdAAcws5EE8+EXZGYNwKnAL7ocoVSd5mZYkJlub8GC9q38KCZPa9u6z+rY\nyg9Dk45J2oRJ+N8FngT2MrOXgPuASSHPfwvwPTLfDnIxs3Fm1mhmja2trSFPK5UsXw+bKCZPe/LJ\nra37rA0bNOmYSCGheumYWXdgH8CAhe5ecJZMMzsNOMXdJ5rZccAV7n5avmPUSycZcY5ADdvDppQY\nouiFU+gc6ukj1SKSXjpm9j+zC3A6QcLfG/iHzLpCjgJON7NFBA9ROd7MHggTlMQrzrp02HnmS4kh\nil44mnRM0qjTFr6ZTctznLv7RaEvohZ+xYh7BGpd3baJEqBbt61191JjiGKEqp4VK7Uikha+u4/N\ns4RO9lJZ4q5Lb9qU+xmrbf/IWmoMLS0wYQL07Bm87tkzGIRVTCIO+6xYjYKVWqKRtimSZF06zlGw\nqq+LbBX1XDpSI5KqS8c9Clb1dZGuUcJPkaQehhHVKNjO6KEeIl2T74+2eXviuPujUQejkk71U7lF\nJFnFlHS659n2D3m2OcH8+CLt5BsF27aWLyLJ0x9tJVJhuzPG/fhBkbSIqoXf9oSnAgcAW6a0cvcf\ndS08qWWvvZa7pNPxc7ztwCu1/EWSEWY+/DuAcwnmzzHgHGBgzHFJlQrTg0aTkomUR5heOke6+/nA\np+7+Q+AI4G/iDUuqVZgeNJqUTKQ8wiT8NZmfX5jZHsAGYHB8IUk+lf5AjkIjVLOt++yHwvr1nbfy\nK/29ilSbMAn/KTP7MnAT8DqwiGAyNCmDan8gRzGDpqr9vYpUmoK9dMxsO3dfl/2d4A+3a7ProqRe\nOvnFPfFZEorpxVPt71UkCVFPrfBy9hd3X+fuK9uuk+TUQu077KRktfBeRSpNvvnwdzOz4UBvMxtm\nZodkluOA+sQiFKC42ne1S9N7FUlSvhb+ycDNQAPwf4GfZJZ/Ar4ff2jSVpomDEvTexVJUpga/tnu\n/kgSwaiG3/kI1DQ9kCNN71WkVFHX8F8ys1+a2e8yJ9/fzC4uKULpVGc9U7K17wkTgqdHTZxYuw/k\n0MNHROIRJuFPA/4A7JF5/TZweWwRpVihEagaoSoipQiT8Hd294eBzQDuvhHYlP8Q6YpCPVPUc0VE\nShEm4X9uZjsRTImMmY0EVsYaVQoV6pminisiUqowCf+7wJPAXmb2EnAfwURqEqFCPVPUc0VEShVq\nPnwz6w7sQzBb5kJ33xBHMGnupVOoZ4p6rohILpHOh29mvYCJwNEEZZ0/mtkd7r42/5FSjEJJW0ld\nREoV5gEo9wGrgH/PvP4GcD/BvPgiIlIlwiT8fdz9oDavnzOzOXEFJCIi8QjzR9s3Mj1zADCzw4GX\nCh1kZr3M7FUzm2NmC8zsh6UEKuFpHnkRySVMwj8c+JOZLTKzRQQzZR5rZvPMbG6e49YBx2e+HRwM\nfL3tB4fER/PIi0guYUo6X+/KiT3o/rM687JHZincJUhK0nE07rXXah55EQkUbOG7+/v5lnzHmlmd\nmTUDy4Bn3P2VHPuMM7NGM2tsbW3t+jsRQKNxRaRzofrhl3yR4BGJjwGT3H1+Z/uluR9+FNo+JSpL\nT4sSqW1Rz5ZZMnf/DJhNF8tDEo5G44pIPrElfDPbJdOyx8x6AycAf47regJPPrl1rp2s9evhiSfK\nE4+IVJYwf7Ttqt2Be82sjuCD5WF3fyrG66WeRuOKSD6xJXx3nwsMi+v8IiJSnERq+CIiUn5K+CIi\nKaGELyKSEkr4IiIpoYQvIpISSvgiIimhhC8ikhJK+CIiKaGELyKSEkr4IiIpoYQvIpISSvgiIimh\nhC8ikhJK+CIiKaGELyKSEkr4IiIpoYQvIpISSvgiIimhhC8ikhJK+CIiKaGELyKSEkr4IiIpoYQv\nIpISSvgiIikRW8I3s78xs+fM7C0zW2Bml8V1LRERKax7jOfeCPxvd3/dzPoCTWb2jLu/GeM1RUSk\nE7G18N19qbu/nvl9FfAW0D+u64mISH6J1PDNbBAwDHglx7ZxZtZoZo2tra1JhCMikkqxJ3wz6wM8\nAlzu7n/tuN3dp7r7CHcfscsuu8QdTvlMnw6DBkG3bsHP6dOr+zoiUnXirOFjZj0Ikv10d380zmtV\ntOnTYdw4+OKL4PX77wevAcaMqb7riEhVMneP58RmBtwLfOLul4c5ZsSIEd7Y2BhLPGU1aFCQfDsa\nOBAWLaq+64hIxTCzJncfEWbfOEs6RwHfAo43s+bMckqM16tcH3xQ3PpKv46IVKXYSjru/iJgcZ2/\nqgwYkLvlPWBAdV5HRKqSRtom4YYboL6+/br6+mB9NV5HRKqSEn4SxoyBqVODWrpZ8HPq1Oj/kDpm\nDFxwAdTVBa/r6oLXUV9n4kTo3j14L927B6+jpt5GItFz94pZhg8f7lKCBx5wr693h61LfX2wPioT\nJrQ/f3aZMCG6ayTxPkRqBNDoIXNsbL10uqJme+kkJYleOt27w6ZN266vq4ONG6O5hnobiYRWKb10\nJGlJ9NLJlezzre8K9TYSiYUSflQK1ZyjqHufcEJwfHY54YT22zvrjRNlL53s3wfCru+KJN6HSAop\n4UchO8L1/feDinN2hGs26U+cCLffvrUVvGlT8LqYpH/CCfDss+3XPfts+6R/SifDHDpb3xXZkbth\n13eFehuJxEI1/CgUqjlHUfe2PEMasv8Nk6p9T5wY9DLatCl4D+PGwW23RXd+CD4sr7kmKOMMGBAk\ne00PIbIN1fDjkK9kU6jmHLbuXWrZJ0ztu9A11B1SpHaF7c6TxFKx3TILdRMcODB3V8WBA4PtdXW5\nt9fVbb1Goe6OubZll6xevXJv79Ur3DXCdIdUt0yRikIR3TLLnuTbLhWb8Asl9EIJKkySLPSh0LNn\n7u09e249R6EPhULXKPQ+w5wjifstIlsUk/DTUdIptUxRqFRSaCTtbbfB/vu3P3b//dvXvQuVfTZs\nyL29s/X5ztXZ+jAloTDlqbjvt4h0TdhPhiSWWFr4UZQHtt8+d4tz++3DHR+mhV+odR4mhkLn6NYt\n97Zu3YLtUbTwo7jfauGLhIZa+G1cc83WB4JkffFFsD6sNWuKW9/R1KmF13fWCye7vtQYAHr3zr8+\nTHfIQt0yo7jf6pYpEo+wnwxJLLG08M1ytxbNwp+jUMs5iuML7RPFOcLciwceCFrSZsHPXC3zCRO2\ntvTr6tp/U4nifoeNQ0TUwm8nzKjNQjXnMKNL850jzPGF9oniHGHuxZgxQZ/9zZuDn7n6vh91FDQ0\nBN8+GhqC17nO1dk1wggTh4gUpfYT/t/+bf71hUbJQuEyRqFzhBmdWmif447Lvb3t+kLniKJUUui9\nJjHaV0S6JuxXgSSWWEo6UXRFdM9fxghzjnzHR3WNMNcptVRSKA79wVUkUWh65DYKTUnQrVvwM9dx\nmzeHu0YU56iEa0QRR1JxauoFEUBTK7QXRV27kCRmd9xxx+LWx6XQe03iXoQpw4nINmo/4SdR105T\nN8JC7zWJexFF10+RNApb+0li6XINv1BdOu66dlTnyCeq7o5RKPRe03QvRMqMVNXws1/v27b46uvj\neUh4Oemxf1vpXohska4aflodEDUJAAAIPUlEQVS+3qepbFSI7oVIl8SW8M3sbjNbZmbz47oGkNxE\nW+WeJ77QBG1ponsh0iWxlXTM7BhgNXCfux8Y5pgulXSS+HqflrKRiFSdiijpuPsLwCdxnX8L9QoR\nEQml7DV8MxtnZo1m1tja2lr8CZL4eq/52UWkBsTaS8fMBgFPxVrSSYJ6hYhIhaqIkk5NUa8QEakB\nSvhhqFeIiNSA7nGd2MweBI4DdjazFuA6d/9lXNeL3ZgxSvAiUtViS/ju/o24zi0iIsVTSUdEJCWU\n8EVEUkIJX0QkJZTwRURSQglfRCQlKmo+fDNrBXIMaU3MzsDyMl4/LMUZLcUZrWqIsxpihHBxDnT3\nXcKcrKISfrmZWWPYIcrlpDijpTijVQ1xVkOMEH2cKumIiKSEEr6ISEoo4bc3tdwBhKQ4o6U4o1UN\ncVZDjBBxnKrhi4ikhFr4IiIpkdqEb2Z1ZvaGmT2VY9uFZtZqZs2Z5ZIyxbjIzOZlYtjmyTAW+LmZ\nvWtmc83skAqN8zgzW9nmfv6gTHF+2cxmmtmfzewtMzuiw/ay388QMZb9XprZPm2u32xmfzWzyzvs\nUwn3MkycZb+fmTj+ycwWmNl8M3vQzHp12L6dmc3I3M9XMg+XKlpss2VWgcuAt4B+nWyf4e6XJhhP\nZ77q7p31w/174O8yy+HA7Zmf5ZAvToA/uvtpiUWT28+A37v7KDPrCXR4qk1F3M9CMUKZ76W7LwQO\nhqDhBCwGHuuwW9nvZcg4ocz308z6A98B9nf3NWb2MDAauKfNbhcDn7r735rZaOBfgXOLvVYqW/hm\n1gCcCvyi3LGU6AzgPg/8F/BlM9u93EFVIjPrBxwD/BLA3de7+2cddivr/QwZY6X5GvDf7t5xwGSl\n/dvsLM5K0R3obWbdCT7kl3TYfgZwb+b3mcDXzMyKvUgqEz5wC/A9YHOefc7OfBWdaWZ/k1BcHTnw\ntJk1mdm4HNv7Ax+2ed2SWZe0QnECHGFmc8zsd2Z2QJLBZewJtALTMqW8X5jZ9h32Kff9DBMjlP9e\ntjUaeDDH+nLfy446ixPKfD/dfTFwM/ABsBRY6e5Pd9hty/10943ASmCnYq+VuoRvZqcBy9y9Kc9u\nvwEGuftQYBZbP1mTdpS7H0Lw9fh/mdkxHbbn+oQvR7erQnG+TjD8+yDg34HHkw6QoAV1CHC7uw8D\nPgeu6rBPue9nmBgr4V4CkCk5nQ78OtfmHOvK0iWwQJxlv59mtgNBC34wsAewvZmd13G3HIcWfT9T\nl/CBo4DTzWwR8BBwvJk90HYHd1/h7usyL+8Chicb4pY4lmR+LiOoPR7WYZcWoO23jwa2/SoYu0Jx\nuvtf3X115vf/AHqY2c4Jh9kCtLj7K5nXMwmSa8d9ynk/C8ZYIfcy6++B19394xzbyn0v2+o0zgq5\nnycAf3H3VnffADwKHNlhny33M1P2+RLwSbEXSl3Cd/er3b3B3QcRfM37T3dv92naodZ4OsEfdxNl\nZtubWd/s78BJwPwOuz0JnJ/pETGS4Kvg0kqL08x2y9Ybzewwgn93K5KM090/Aj40s30yq74GvNlh\nt7LezzAxVsK9bOMbdF4mKfu/zTY6jbNC7ucHwEgzq8/E8jW2zTlPAhdkfh9FkLeKbuGnuZdOO2b2\nI6DR3Z8EvmNmpwMbCT5FLyxDSLsCj2X+LXYHfuXuvzez8QDufgfwH8ApwLvAF8DYCo1zFDDBzDYC\na4DRXfnHGoFJwPTMV/z3gLEVeD8LxVgR99LM6oETgW+3WVdp9zJMnGW/n+7+ipnNJCgvbQTeAKZ2\nyEm/BO43s3cJctLorlxLI21FRFIidSUdEZG0UsIXEUkJJXwRkZRQwhcRSQklfBGRlFDCF8nIzJyY\na/bUnOsjuN6ZZrZ/m9ezzazin7Mq1UsJX6R8zgT2L7iXSESU8KVqZEb1/jYz0dV8Mzs3s364mT2f\nmbztD9mR0pkW8y1m9qfM/odl1h+WWfdG5uc++a6bI4a7zey1zPFnZNZfaGaPmtnvzewdM/u3Nsdc\nbGZvZ+K5y8z+n5kdSTCK+yYL5mHfK7P7OWb2amb//xHRrRMBNNJWqsvXgSXufiqAmX3JzHoQTHp1\nhru3Zj4EbgAuyhyzvbsfmZnQ7W7gQODPwDHuvtHMTgB+DJwdMoZrCIa1X2RmXwZeNbNZmW0HA8OA\ndcBCM/t3YBNwLcGcOKuA/wTmuPufzOxJ4Cl3n5l5PwDd3f0wMzsFuI5gnhWRSCjhSzWZB9xsZv9K\nkCj/aGYHEiTxZzIJs45gitmsBwHc/QUz65dJ0n2Be83s7whmHOxRRAwnEUy+d0XmdS9gQOb3Z919\nJYCZvQkMBHYGnnf3TzLrfw3snef8j2Z+NgGDiohLpCAlfKka7v62mQ0nmKPlRjN7mmB2zgXufkRn\nh+V4PQV4zt3PsuBRcbOLCMOAszNPU9q60uxwgpZ91iaC/7+KfUhF9hzZ40Uioxq+VA0z2wP4wt0f\nIHhgxCHAQmAXyzz71cx6WPuHWGTr/EcTzNi4kmBq2cWZ7RcWGcYfgEltZlgcVmD/V4FjzWwHC6a1\nbVs6WkXwbUMkEWpBSDUZQvBHzs3ABmCCu683s1HAz83sSwT/pm8BFmSO+dTM/kTw7OJsXf/fCEo6\n3yWoqRdjSub8czNJfxHQ6fNQ3X2xmf0YeIVgPvg3CZ5WBMHzGO4ys+8QzNooEivNlik1y8xmA1e4\ne2OZ4+jj7qszLfzHgLvdPdfDtEVipZKOSPwmm1kzwYNh/kIZH0so6aYWvohISqiFLyKSEkr4IiIp\noYQvIpISSvgiIimhhC8ikhJK+CIiKfH/ATx2ayG3VuqtAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0xa4d0ef0>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "import matplotlib.pyplot as plt\n",
    "\n",
    "plt.scatter(X_setosa[:, 0], X_setosa[:, 2], color='red', marker='o', label='setosa')\n",
    "plt.scatter(X_versicolor[:, 0], X_versicolor[:, 2], color='blue', marker='^', label='versicolor')\n",
    "plt.scatter(X_virginica[:, 0], X_virginica[:, 2], color='green', marker='s', label='virginica')\n",
    "plt.xlabel('sepal length')\n",
    "plt.ylabel('petal length')\n",
    "plt.legend(loc = 'upper left')\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "由上图可见，三个类别之间是有较明显区别的。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 训练集、验证集、测试集划分"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "接下来，我们要将每个类别的所有样本分成**训练样本**（training set）、**验证集(validation set)**和**测试样本**（test set），各占所有样本的比例分别为60%，20%，20%。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 66,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "# training set\n",
    "X_setosa_train = X_setosa[:30, :]\n",
    "y_setosa_train = y_setosa[:30]\n",
    "X_versicolor_train = X_versicolor[:30, :]\n",
    "y_versicolor_train = y_versicolor[:30]\n",
    "X_virginica_train = X_virginica[:30, :]\n",
    "y_virginica_train = y_virginica[:30]\n",
    "X_train = np.vstack([X_setosa_train, X_versicolor_train, X_virginica_train])\n",
    "y_train = np.hstack([y_setosa_train, y_versicolor_train, y_virginica_train])\n",
    "\n",
    "# validation set\n",
    "X_setosa_val = X_setosa[30:40, :]\n",
    "y_setosa_val = y_setosa[30:40]\n",
    "X_versicolor_val = X_versicolor[30:40, :]\n",
    "y_versicolor_val = y_versicolor[30:40]\n",
    "X_virginica_val = X_virginica[30:40, :]\n",
    "y_virginica_val = y_virginica[30:40]\n",
    "X_val = np.vstack([X_setosa_val, X_versicolor_val, X_virginica_val])\n",
    "y_val = np.hstack([y_setosa_val, y_versicolor_val, y_virginica_val])\n",
    "\n",
    "# test set\n",
    "X_setosa_test = X_setosa[40:50, :]\n",
    "y_setosa_test = y_setosa[40:50]\n",
    "X_versicolor_test = X_versicolor[40:50, :]\n",
    "y_versicolor_test = y_versicolor[40:50]\n",
    "X_virginica_test = X_virginica[40:50, :]\n",
    "y_virginica_test = y_virginica[40:50]\n",
    "X_test = np.vstack([X_setosa_test, X_versicolor_test, X_virginica_test])\n",
    "y_test = np.hstack([y_setosa_test, y_versicolor_test, y_virginica_test])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 4. KNN训练函数和预测函数"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "KNN的训练过程实际上是一种数据标类、数据存储的过程，不包含机器学习算法。首先我们需要定义一个**类**（class）来实现KNN算法模块。该类的初始化定义为："
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "```python\n",
    "class KNearestNeighbor(object):\n",
    "    def __init__(self):\n",
    "        pass\n",
    "```"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "然后，在KNearestNeighbor类中定义训练函数，训练函数保存所有训练样本。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "```python\n",
    "def train(self, X, y):\n",
    "    self.X_train = X\n",
    "    self.y_train = y\n",
    "    \n",
    "```"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "KNN的测试过程是核心部分。其中，有两点需要注意：\n",
    "\n",
    "* 衡量距离的方式\n",
    "\n",
    "* K值的选择"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 衡量距离的方式"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "KNN距离衡量一般有两种方式：**L1距离**和**L2距离**。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "L1距离的计算公式为："
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "$$d1(I_1,I_2)=\\sum_p|I_1^p-I_2^p|$$"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "其中，$I_1$和$I_2$分别表示两个样本，$p$表示特征维度。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "L2距离的计算公式为："
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "$$d2(I_1,I_2)=\\sqrt{\\sum_p(I_1^p-I_2^p)^2}$$"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "一般来说，L1距离和L2距离都比较常用。需要注意的是，如果两个样本距离越大，那么使用L2会继续扩大距离，即对距离大的情况**惩罚性越大**。反过来说，如果两个样本距离较小，那么使用L2会缩小距离，减小惩罚。这里，我们使用最常用的L2距离。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### K值的选择"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "KNN中K值的选择至关重要，K值太小会使模型过于复杂，造成**过拟合**（overfitting）；K值太大会使模型分类模糊，造成**欠拟合**(underfitting)。实际应用中，我们可以选择不同的K值，通过验证来决定K值大小。代码中，我们将K设定为可调参数。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "在KNearestNeighbor类中定义预测函数："
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "```python\n",
    "def predict(self, X, k=1)\n",
    "    # 计算L2距离\n",
    "    num_test = X.shape[0]\n",
    "    num_train = self.X_train.shape[0]\n",
    "    dists = np.zeros((num_test, num_train))    # 初始化距离函数\n",
    "    # because(X - X_train)*(X - X_train) = -2X*X_train + X*X + X_train*X_train, so\n",
    "    d1 = -2 * np.dot(X, self.X_train.T)    # shape (num_test, num_train)\n",
    "    d2 = np.sum(np.square(X), axis=1, keepdims=True)    # shape (num_test, 1)\n",
    "    d3 = np.sum(np.square(self.X_train), axis=1)    # shape (1, num_train)\n",
    "    dist = np.sqrt(d1 + d2 + d3)\n",
    "    # 根据K值，选择最可能属于的类别\n",
    "    y_pred = np.zeros(num_test)\n",
    "    for i in range(num_test):\n",
    "        dist_k_min = np.argsort(dist[i])[:k]    # 最近邻k个实例位置\n",
    "        y_kclose = self.y_train[dist_k_min]     # 最近邻k个实例对应的标签\n",
    "        y_pred[i] = np.argmax(np.bincount(y_kclose))    # 找出k个标签中从属类别最多的作为预测类别\n",
    "    \n",
    "    return y_pred\n",
    "```"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "KNearestNeighbor类的完整定义代码如下："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 154,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "class KNearestNeighbor(object):\n",
    "    def __init__(self):\n",
    "        pass\n",
    "\n",
    "    # 训练函数\n",
    "    def train(self, X, y):\n",
    "        self.X_train = X\n",
    "        self.y_train = y\n",
    "    \n",
    "    # 预测函数\n",
    "    def predict(self, X, k=1):\n",
    "        # 计算L2距离\n",
    "        num_test = X.shape[0]\n",
    "        num_train = self.X_train.shape[0]\n",
    "        dists = np.zeros((num_test, num_train))    # 初始化距离函数\n",
    "        # because(X - X_train)*(X - X_train) = -2X*X_train + X*X + X_train*X_train, so\n",
    "        d1 = -2 * np.dot(X, self.X_train.T)    # shape (num_test, num_train)\n",
    "        d2 = np.sum(np.square(X), axis=1, keepdims=True)    # shape (num_test, 1)\n",
    "        d3 = np.sum(np.square(self.X_train), axis=1)    # shape (1, num_train)\n",
    "        dist = np.sqrt(d1 + d2 + d3)\n",
    "        # 根据K值，选择最可能属于的类别\n",
    "        y_pred = np.zeros(num_test)\n",
    "        for i in range(num_test):\n",
    "            dist_k_min = np.argsort(dist[i])[:k]    # 最近邻k个实例位置\n",
    "            y_kclose = self.y_train[dist_k_min]     # 最近邻k个实例对应的标签\n",
    "            y_pred[i] = np.argmax(np.bincount(y_kclose.tolist()))    # 找出k个标签中从属类别最多的作为预测类别\n",
    "\n",
    "        return y_pred\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 5. 训练和预测"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 选择合适的K值 "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "首先，创建一个KnearestNeighbor实例对象。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 155,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "KNN = KNearestNeighbor()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "然后，在验证集上进行k-fold交叉验证。选择不同的K值，根据验证结果，选择最佳的K值。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 176,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "k = 3, accuracy: 0.888889\n",
      "k = 5, accuracy: 0.877778\n",
      "k = 7, accuracy: 0.888889\n",
      "k = 9, accuracy: 0.888889\n",
      "k = 11, accuracy: 0.866667\n",
      "k = 13, accuracy: 0.855556\n",
      "k = 15, accuracy: 0.877778\n",
      "Best K is: 3\n",
      "\n"
     ]
    }
   ],
   "source": [
    "num_folds = 5    # 训练数据分为5 folds\n",
    "K_classes = [3, 5, 7, 9, 11, 13, 15]    # 所有K值\n",
    "\n",
    "# 把训练数据分成5份\n",
    "X_train_folds = []\n",
    "y_train_folds = []\n",
    "X_train_folds = np.split(X_train, num_folds)\n",
    "y_train_folds = np.split(y_train, num_folds)\n",
    "\n",
    "# 字典用来存储不同K值对应的准确率\n",
    "K_accuracy = []\n",
    "k_best = K_classes[0]\n",
    "\n",
    "for k in K_classes:\n",
    "    accuracies = []\n",
    "    for i in range(num_folds):\n",
    "        Xtr = np.concatenate(X_train_folds[:i] + X_train_folds[i+1:])\n",
    "        ytr = np.concatenate(y_train_folds[:i] + y_train_folds[i+1:])\n",
    "        Xcv = X_train_folds[i]\n",
    "        ycv = y_train_folds[i]\n",
    "        KNN.train(Xtr, ytr)\n",
    "        ycv_pred = KNN.predict(Xcv, k=k)\n",
    "        accuracy = np.mean(ycv_pred == ycv)\n",
    "        accuracies.append(accuracy)\n",
    "    accuracies_avg = np.mean(accuracies)\n",
    "    K_accuracy.append(accuracies_avg)\n",
    "    if accuracies_avg > k_best:\n",
    "        k_best = accuracies_avg\n",
    "\n",
    "# 打印出验证结果\n",
    "for k in range(len(K_classes)):\n",
    "    print('k = %d, accuracy: %f' % (K_classes[k], K_accuracy[k]))\n",
    "print('Best K is: %d\\n' % k_best)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "作图，查看验证结果。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 180,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAZIAAAEWCAYAAABMoxE0AAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMS4wLCBo\ndHRwOi8vbWF0cGxvdGxpYi5vcmcvpW3flQAAIABJREFUeJzt3XeYVOX5//H3BxAQBBuLCkgxooIl\nlhXFqFGJithjAxYbfOX7VdHEmF80igZJTIzGlsSYYFhAmkE0hiiWxNiDZUFFwYYGccGytoAgkXL/\n/njOhHGZ3T3s7syZcr+ua66dOXPmzH0Wdu45T7kfmRnOOedcY7VIOgDnnHOFzROJc865JvFE4pxz\nrkk8kTjnnGsSTyTOOeeaxBOJc865JvFE4lwWSBojaUp0v7ukLyS1bGjfRr7XAkmHNfb1uSDJJO2c\ndBwuOzyRuLwhaaikquhD931JD0o6OOm4msrMlpjZFma2rqnHkjRR0s9qHX93M3u8qcd2rrE8kbi8\nIOkHwC3Az4HtgO7A74AT69i/Ve6ic87VxxOJS5ykLYGxwIVmdq+ZrTSzNWb2VzP7f9E+YyTNlDRF\n0nLgHEltJN0iaVl0u0VSm2j/TpLul/S5pE8lPSWpRfTcZZKWSloh6Q1JA+qI6yFJo2pte1nSd6P7\nt0p6T9JySXMlHVLHcXpGTTutose9JD0Rvf/fgE619r9b0geS/i3pSUm7R9tHAhXAj6Krtr9G2xdL\n+k50v77fyWGSqiVdKumj6Krv3Hr+XbpImhX9/hZJOi/tuTGSZki6MzqPBZLK6zpWreMeHP3eDo+z\nv8t/nkhcPugPtAX+3MB+JwIzga2AqcCVwIHA3sA3gX7A6GjfS4FqoIxwhXMFYJJ2BUYB+5tZB+Bo\nYHEd7zcNGJJ6IKkv0AN4INr0QvTe20T73i2pbYzznQbMJSSQnwJn13r+QaA30BmYF50rZjYuun99\n1FR2fIZj1/c7Adge2BLoCowAbpO0dR1xTif8DrsApwI/r5V0TwDuIvx7zAJ+29CJSzo6Ou4pZvZY\nQ/u7wuCJxOWDbYGPzWxtA/vNMbP7zGy9mX1J+HY+1sw+MrMa4BrgzGjfNcAOQI/o6uYpC4Xl1gFt\ngL6SNjOzxWb2dh3v92dgb0k9oscVwL1m9h8AM5tiZp+Y2VozuzE67q71nYCk7sD+wFVm9h8zexL4\na/o+ZlZpZiui9xkDfDO6aoujvt9J6vcyNvqdzAa+yBSzpB2Bg4HLzGy1mb0E/LHWsZ42s9lR389k\nQuKqz2nAOGCQmT0f83xcAfBE4vLBJ0CnGP0e79V63AV4N+3xu9E2gBuARcAjkt6RdDmAmS0Cvk/4\ngP5I0l2SugBEzUWpW3czW0G4+hgcHXMw0dVBtP+lkl6LmqA+J3zT/1ozVQZdgM/MbGWtuFPHbCnp\nOklvR014i6OnGjpu+vHr+p0AfFIrYa8CtqjjOJ9Gv4P0Y3VNe/xBreO0beDf8PvADDN7pZ59XAHy\nROLywRxgNXBSA/vVLlW9jNDUlNI92kb0jf5SM9sJOB74QapZxsymmdnB0WsN+GW0fYu025LomNOB\nIZL6A5sDjwFE/SGXAacDW5vZVsC/ATVwDu8DW0tqXyvulKGEJrzvEBJTz2h76rgNleuu83eyiZYB\n20jqUOtYSxtxrJTTgJMkfb8Jx3B5yBOJS5yZ/Ru4mtBef5KkdpI2k3SMpOvreel0YLSkMkmdomOk\n5m4cJ2lnSQKWE5q01knaVdIRUQf0auDL6Lm6zCZ8MI8F/mRm66PtHYC1QA3QStLVQMcY5/ouUAVc\nI6m1wvDm9L6ODsB/CFdp7Qij2NJ9COxUz1vU+TvZFGb2HvBP4BeS2krai9CnMrX+V9ZrGTAAuFjS\nBU04jssznkhcXjCzm4AfEDqGawjNWKOA++p52c8IH8rzgVcIHdOpORa9gb8T+gDmAL+L5lq0Aa4D\nPiY0zXQmdMTXFdd/gHsJVwjT0p56mNAp/iahyWc1Gze91WUocADwKfAT4M605+6MjrcUWAg8W+u1\n4wn9O59LyvS7qe93sqmGEK6IlhH6i35iZn9r5LGAMKeGkEwuk/Q/TTmWyx/yha2cc841hV+ROOec\naxJPJM4555rEE4lzzrkm8UTinHOuSbJa+E7SQOBWoCXwRzO7rtbz3YFJhBILLYHLzWy2pNbAH4By\nYD3wvVR1U0n7ARMJY/pnR8/VO2KgU6dO1rNnz+Y7MeecKwFz58792MzKGtova4lEYe2F24AjCfV6\nXpA0y8wWpu02mjDT9faojtFswnDD8wDMbE9JnYEHJe0fjeG/HRhJGBY5GxhIGIZZp549e1JVVdWs\n5+ecc8VO0rsN75Xdpq1+wCIze8fMviIUd6tdEtzYMIlrSzbMwO0LPApgZh8BnwPlknYAOprZnOgq\n5E4ang3tnHMui7KZSLry9Qla1Xy9Tg+EekfDJFUTri4uira/DJwoqZWkXsB+wI7R66sbOKZzzrkc\nymYiyVRzqHZfxhBgopl1AwYBkxXWjKgkJIkqwmJH/ySUo4hzzPDm0kiF1faqampqGnkKzjnnGpLN\nzvZqwlVESjc2Lh43gtDHgZnNidZy6BQ1Z12S2knSP4G3gM+i49R3TKLjjSOUrKa8vNyn7zvnXJZk\n84rkBaB3tBpca0IJ7lm19knV3UFSH8LiRjVR0b720fYjgbVmttDM3gdWSDowKsZ3FvCXLJ6Dc865\nBmQtkURrHowiFLd7jTA6a4GksZJOiHa7FDhP0suEqqXnRJ3onYF5kl4jlOpOX0znfMICO4uAt2lg\nxFajTZ0KPXtCixbh59SmFD11zaKY/k2K6VxcySuJoo3l5eW2ScN/p06FkSNh1aoN29q1g3HjoKKi\n+QN0DSumf5NiOhdX1CTNNbPyBvfzRJJBz57wbobh0z16wOLFzRWW2xR1/ZtsuSVcfHHOw2mSX/8a\n/v3vjbf7/y+XZzyRpNnkRNKiBWT6vUiwfv3G21321fVvAuHfpZDUdx7+/8vlkbiJxGttZdK9+6Zt\nd9lX1+++R4/w4VtItx49Mp+L//9yBcoTSSbXXhvarNO1ahW2u2SclKGAQbt2hflvkun/V6Gei3N4\nIsmsoiJ0fPboEZobOnaEtWvDzeXeW2/BhAmw007hW7sU/m0KtXM6/f9XypgxhXkuzuGJpG4VFaHj\nc/16+OQTOPxw+L//g/nzk46stKxaBaecEq4I//GP0OG+fn34tynkD97U/68PPgjn9tFHSUfkXKN5\nIomjVSuYPh223jp8qGUaceOanxlccAG8+moYMltX30Ih2247OO44uPNOWLMm6WicaxRPJHFttx3M\nmAH/+hcMH173yBvXfP74R5g0Ca6+GgYOTDqa7BkxIlyRPPBA0pE41yieSDbFwQfD9dfDvffCTTcl\nHU1xmzcPLroIjjoKrroq6Wiya+BA2GEHGD8+6UicaxRPJJvqkktC89Zll8FTTyUdTXH69NPwO+7c\nOTRptWyZdETZ1aoVnH02zJ4NyzLWIHUur3ki2VQSVFaGEUSnnx46S13zWb8ezjoLli6Fu++GTp2S\njig3hg8P537nnUlH4twm80TSGB07wj33hE73wYN9WHBzuu660Fdw001wwAFJR5M7vXvDIYeELyne\n/+YKjCeSxtpzT/jDH+CJJ2D06KSjKQ6PPhr6QwYPhgsvTDqa3BsxIsyZefrppCNxbpN4ImmKM8+E\n//1f+OUvYVbtpVbcJlm6FIYMgV13hTvuKLz6Wc3h1FOhQ4dwVeJcAfFE0lS33AL77Rfa9d9+O+lo\nCtOaNXDGGWHy4T33wBZbJB1RMtq3D1djM2bA8uVJR+NcbJ5ImqptW5g5M1SnPfVU+PLLpCMqPJdd\nBs88E+aN9OmTdDTJGj48JNQZM5KOxLnYPJE0h549YfJkeOmlMPfBxTdzJtx8M4waFb6Nl7oDDoC+\nfX1OiSsonkiay7HHwpVXhg+ACROSjqYwvPlm+AZ+wAFw441JR5MfpPA7efZZWLgw6Wici8UTSXO6\n5hoYMCDUh3rppaSjyW8rV4ZJh61bh2ac1q2Tjih/nHlmmKTone6uQHgiaU4tW8K0abDttqG/5PPP\nk44oP5nB+efDggXh9+ULOn1d585wwgleyNEVDE8kza1z5/AN+9134dxzfXJZJuPGhT6lMWNCLS23\nseHDoaYG7r8/6Uica1BWE4mkgZLekLRI0uUZnu8u6TFJL0qaL2lQtH0zSZMkvSLpNUk/TnvN4mj7\nS5I2YSH2HDroILjhBrjvPvjVr5KOJr9UVcHFF4dChT6Rs25HHw1duninuysIWUskkloCtwHHAH2B\nIZL61tptNDDDzPYBBgO/i7afBrQxsz2B/YD/ldQz7XWHm9necRalT8z3vgennQaXXx5mv7uwQNip\np8L228OUKWHItMssVcjxwQe9kKPLe9n8S+4HLDKzd8zsK+Au4MRa+xjQMbq/JbAsbXt7Sa2AzYGv\ngMKaoSWFeRE77xwm273/ftIRJWv9+tCJvGxZKMa47bZJR5T/UoUcJ01KOhLn6pXNRNIVeC/tcXW0\nLd0YYJikamA2kJqEMRNYCbwPLAF+ZWafRs8Z8IikuZJG1vXmkkZKqpJUVVNT0+STaZRUcccVK7y4\n489/Hr5d33IL9OuXdDSFYeed4dBDvZCjy3vZTCSZiiXV/msYAkw0s27AIGCypBaEq5l1QBegF3Cp\npJ2i13zLzPYlNJldKOnQTG9uZuPMrNzMysvKyprhdBppjz1C5/KTT8IVVyQXR5L+/vewyuHQoWG0\nlotvxAhYtMjXvnF5LZuJpBrYMe1xNzY0XaWMAGYAmNkcoC3QCRgKPGRma8zsI+AZoDzab1n08yPg\nz4Skk98qKsIHaKoDvpRUV4dijH36hIRaisUYm+KUU7yQo8t72UwkLwC9JfWS1JrQmV67RO4SYACA\npD6ERFITbT9CQXvgQOB1Se0ldYj2bw8cBbyaxXNoPjffDPvvHzpQFy1KOprc+OqrsPjX6tWhia99\n+6QjKjzt24dEfPfdXsjR5a2sJRIzWwuMAh4GXiOMzlogaaykE6LdLgXOk/QyMB04x8yMMNprC0KS\neAGYYGbzge2Ap6P9nwceMLOHsnUOzapNm/Bh0KpV6RR3/NGPYM6cMIR1t92SjqZwpQo5/ulPSUfi\nXEayEujEKy8vt6qqPJly8uCDoS7XOecUd3PFjBlhtNrFF8OttyYdTWEzCwupbbFFqMHlXI5Imhtn\nmoUP5M+1Y44JE/EmTCjeyWavvx46ifv3D/1Crmmk8Pt87rlQVsa5POOJJAk/+QkceWRYTvbFF5OO\npnmtXBma7tq29WKMzWnYMNhss+K+inUFyxNJElq2hKlToaysuIo7moWlhxcuhOnToVu3pCMqHmVl\noZDj5MlhEINzecQTSVLKysI39iVLwkiu9euTjqjpfv/7kCDHjoXvfCfpaIqPF3J0ecoTSZL69w8L\nOs2aVfh9Cc8/D9//PgwaVLoTL7Pt6KOha9fi7VtzBcsTSdIuuijMtbjiCnjssaSjaZxPPgkFKnfY\nITS9eDHG7GjZMly9PvQQLF2adDTO/Zf/xSctVdxxl11CPa5Cq/S6fn3oCP7gg7D++jbbJB1RcTv3\nXC/k6PJOg4lE0nFR/SuXLR06hA/hL74Icy8KaVW8n/0sfEO+9VYoz9+q/kVj553h29/2Qo4ur8RJ\nEIOBtyRdH5Uxcdmw++5wxx3w9NPw4x83vH8+eOSRsMrhsGFhtJbLjREj4O23QyFQ5/JAg4nEzIYB\n+wBvAxMkzYlKtHfIenSlZujQMLfkxhvh3nuTjqZ+770X4u3bN4zW8mKMuXPKKWGJAp9T4vJErCYr\nM1sO3ENYnGoH4GRgnqSL6n2h23Q33hjW6zj3XHjrraSjyeyrr0Ln+ldfeTHGJLRrt6GQ47//nXQ0\nzsXqIzle0p+BfwCbAf3M7Bjgm8APsxxf6UkVd9xss/DNc9WqpCPa2A9/GMp1VFbCrrsmHU1pGjEi\nFP70Qo4uD8S5IjkNuNnM9jKzG6J1QDCzVcDwrEZXqrp3DxP7Xn0VLrggvzpV77oLfvObMGfk1FOT\njqZ0lZeHRdN8TonLA3ESyU8IJdsBkLS5pJ4AZvZodsJyHH10WFVw0qQwPDgfvPYa/M//wLe+Bddf\nn3Q0pS1VyPH558MXDucSFCeR3A2k1+9YF21z2XbVVXDUUWHS4rx5ycbyxRehqa19+9Ccstlmycbj\nvJCjyxtxEkkrM/tvlbjovpd0zYVUccfOncOH+GefJROHGYwcCW+8EYoxdu2aTBzu6zp1ghNP9EKO\nLnFxEklN2oqGSDoR+Dh7Ibmv6dQpdL4vXQpnnZVMccff/S4kkJ/+FI44Ivfv7+o2fDh8/DH89a9J\nR+JKWJxE8n/AFZKWSHoPuAzw2We5dMABcNNNoerrL3+Z2/d+7jm45BI47ji4/PLcvrdr2FFHeSFH\nl7g4ExLfNrMDgb5AXzM7yMwWZT809zUXXhhqcY0eDY/maIzDxx+H+SJdu8Kdd3oxxnzUsmVYtvnh\nh6G6OuloXImK9ckg6VjgAuASSVdLujq7YbmNSKGEyq67hslo2a7+um4dVFTAhx+GOmBbb53d93ON\n54UcXcLiTEj8PXAGcBEgwrySHlmOy2WyxRZhJvmqVdkv7vjTn4ZaWr/5Dey3X/bexzXdN74Bhx0W\nRm8VwwJpruDEuSI5yMzOAj4zs2uA/sCOcQ4uaaCkNyQtkrRRA7uk7pIek/SipPmSBkXbN5M0SdIr\nkl6T9OO4xyx6ffqEeSXPPAOXXZad93joobDK4VlnwXnnZec9XPMaMQLeeccLObpExEkkq6OfqyR1\nAdYAvRp6kaSWwG3AMYT+lSGS+tbabTQww8z2IVQZ/l20/TSgjZntCewH/K+knjGPWfwGD4ZRo+Dm\nm0OzU3NasiQ0ae2xB9x+uxdjLBSnnAJbbulzSlwi4iSSv0raCrgBmAcsBqbHeF0/YJGZvRPNPbkL\nOLHWPgZ0jO5vCSxL295eUitgc+ArYHnMY5aGG28Mo7mGD4c332yeY/7nP6Fzfc2akKDatWue47rs\n23zz0Hc2c6YXcnQ5V28iiRa0etTMPjezewh9I7uZWZzO9q7Ae2mPq6Nt6cYAwyRVA7MJ/TAAM4GV\nwPvAEuBXZvZpzGOmYh8pqUpSVU1NTYxwC0zr1mF+SZs24dvoypVNP+all4aSGxMnhhUbXWFJFXK8\n666kI3Elpt5EYmbrgRvTHv/HzOJ+3cnUJlK7+uAQYKKZdQMGAZOj5NWPUIqlC6EZ7VJJO8U8ZirW\ncWZWbmblZWVlMUMuMDvuCNOmwYIFcP75TSvuOG0a3HZbSCbf/W7zxehyZ7/9YM89fU6Jy7k4TVuP\nSDpF2uTG8mq+3infjQ1NVykjgBkAZjYHaAt0AoYCD5nZmqja8DNAecxjlpYjjwyrFE6eDOPGNe4Y\nCxeGTvWDD4Zf/KJZw3M5lCrk+MIL8MorSUfjSkicRPIDQpHG/0haLmmFpOUxXvcC0FtSL0mtCZ3p\ns2rtswQYABAt49sWqIm2H6GgPXAg8HrMY5ae0aNh4EC4+GKoqtq0165YEZrGOnTwYozFoKLCCzm6\nnIszs72DmbUws9Zm1jF63DHG69YCo4CHgdcIo7MWSBqbVrvrUuA8SS8TOvDPMTMjjMzaAniVkDwm\nmNn8uo65yWddbFq0gClTYPvtwxohn34a73Vm4UrkzTdDu3qXLtmN02Vfp05w0kleyNHllKyBdnVJ\nh2babmYFM2C9vLzcqjb1m3ohev750Dx15JGhiF9DJU1+85twFfOLX3gdrWLy0ENwzDFhMIYvPuaa\nQNJcMytvcL8YiSS9rGhbQkf4XDMrmDKwJZNIIFTqvfDCMDN99Oi695szBw49NHzg3Hef19EqJuvW\nQc+eYS7Qgw8mHY0rYHETSZymrePTbkcCewAfNkeQLgvOPx+GDg2rK/7975n3qamB008Po74mTfIk\nUmzSCzm+916DuzvXVI35BKkmJBOXj6QweqtPnzBBrXZF2HXrQqKpqQl1u7wYY3E699zQB+aFHF0O\nxCna+BtJv45uvwWeAl7Ofmiu0dq3D0li9epw5ZHe6XrNNeFK5be/hX32SS5Gl1077QSHHw4TJngh\nR5d1ca5IqoC50W0OcJmZDctqVK7pdtstTEybMwdOOCG0mbdoEfpODjkkzDdwxS1VyPGJJ5KOxOXa\n1Kkb/uZ79gyPsyhOZ3t7YLWZrYsetyQUVFyV1ciaUUl1ttc2cGBoK0+3+eZhbZOKimRicrnx5Zew\nww5w/PFhOLArDVOnwsiRYbmJlHbtQpP3Jv7NN1tnO/AooXBiyuZAHb24Lu+89trG2778Eq68Mvex\nuNzafPPQHzZzJnz+edLRuFy58sqvJxEIj7P4Nx8nkbQ1sy9SD6L7Xha2UNQ1amfJktzG4ZIxYkTo\nK/NCjqWjrr/tLP7Nx0kkKyXtm3ogaT/gy6xF5JpX9+6btt0Vl333hb328kKOpSSBv/k4ieT7wN2S\nnpL0FPAnQpkSVwiuvXbjdUXatQvbXfFLFXKsqoL585OOxuXCtdduXDMvy3/zcSYkvgDsBpwPXAD0\nMbO5WYvINa+KitDJ1qNH+FDp0aNRnW6ugFVUhPVrvJBjaRg6FLbbLvyb5+hvPs48kguB9mb2qpm9\nAmwh6YKsReSaX0UFLF4c5hMsXuxJpNRsu20o5DhlSlgF0xW3qqowEfnXv87Z33ycpq3zzOy/Qz7M\n7DPgvOyF5JxrdsOHwyefwCxfdaHoVVaGEXuDB+fsLeMkkhbpi1pF80haZy8k51yz+853Qm0173Qv\nbqtWhdVOTz0VttwyZ28bJ5E8DMyQNEDSEYR1Qx7KbljOuWaVKuT4yCNeyLGY3XsvLF+e88oVcRLJ\nZcA/CJ3tFxImKP4om0E557IgVchx4sSkI3HZMn48fOMbYYmIHIozamu9md1uZqea2Slm9odUuRTn\nXAHp1QsGDPBCjsXq7bfh8cdDf9iG3oiciDNqq7ekmZIWSnondctFcM65ZjZ8OPzrX+EDxxWXCRNC\nkcazz875W8dp2poA3A6sBQ4H7gS8Apxzhejkk2GrrXxOSbFZty40WQ4cCF275vzt4ySSzc3sUUKl\n4HfNbAxQMMvsOufSpAo53nOPF3IsJo88AkuXhivOBMRJJKsltQDekjRK0slA5zgHlzRQ0huSFkm6\nPMPz3SU9JulFSfMlDYq2V0h6Ke22XtLe0XOPR8dMPRcrFudcJFXIcfr0pCNxzaWyEjp1CksGJCBu\nra12wMXAfsAwoMFGuGi+yW3AMUBfYIikvrV2Gw3MMLN9gMHA7wDMbKqZ7W1mewNnAovN7KW011Wk\nnjezj2Kcg3MuZZ994Jvf9DklxaKmBv7yFzjzzFAWJQGxam2Z2RdmVm1m50Yjt56Ncex+wCIze8fM\nvgLuAk6sfXigY3R/S2BZhuMMIcxdcc41h1Qhx7lz4WVfNbvgTZ0Ka9Yk1qwF8a5IGqsrkD7zqTra\nlm4MMExSNTAbuCjDcc5g40QyIWrWuip91r1zLqahQ72QYzEwC1eW/frBHnskFkY2E0mmD/ja6/oO\nASaaWTdgEDA56o8JB5AOAFaZ2atpr6kwsz2BQ6LbmRnfXBopqUpSVU1NTVPOw7nis+22YQSXF3Is\nbC+8AK++mvOZ7LVlM5FUAzumPe7Gxk1XI4AZAGY2B2gLdEp7fjC1rkbMbGn0cwUwjdCEthEzG2dm\n5WZWXlZW1oTTcK5IDR8On34a2tddYUoVaDzjjETDiDMhsUzSFZLGSapM3WIc+wWgt6RekloTkkLt\n0qNLgAHR+/QhJJKa6HEL4DRC30oqllaSOkX3NwOOA17FObfpvvOdsGqed7oXplWrwsi7007LaYHG\nTFrF2OcvwFPA34HYpVHMbK2kUYSijy2BSjNbIGksUGVms4BLgTskXUJo9jrHzFLNX4cC1WaWPou+\nDfBwlERaRjHdETcm51yaFi1C/a2xY8N63r78cmG5555ECjRmog2f23XsIL0UDcMtWOXl5VZVVZV0\nGM7ln8WLYaedYMwYuPrqpKNxm+Kww8IkxDffzFptLUlzzay8of3i9JHcn5oo6JwrMj17eiHHQrRo\nETzxRCIFGjOJk0i+R0gmqyWtiG7Lsx2Ycy5Hhg8PVyaPPZZ0JC6uVIHGs85KOhIg3oTEDmbWwsza\nRvc7mFnHhl7nnCsQXsixsKQKNB5zTCIFGjOJNfxX0gmSfhXdjst2UM65HGrbFioqQuftZ58lHY1r\nyMMPw7Jlic5kry3O8N/rCM1bC6Pb96JtzrliMWJEmJjohRzzX2UllJXBcfnznT7OFckg4EgzqzSz\nSmBgtM05Vyz22Qf23tvnlOS7mhqYNSvRAo2ZxJ3ZvlXa/WRnvjjnsmPECJg3D156qeF9XTKmTEm8\nQGMmcRLJL4AXJU2UNAmYC/w8u2E553Ju6FBo08Y73fNVqkDjAQfA7rsnHc3XxBm1NR04ELg3uvU3\ns7vqf5VzruBss82GQo6rVycdjavt+edhwYK8mMleW52JRNJu0c99gR0IRRjfA7pE25xzxWbEiDBy\nyws55p/KSmjXLvECjZnUV2vrB8BI4MYMzxm+brtzxeeII6BHj9CEkocfWCVr5coNBRo75t80vjoT\niZmNjO4eY2Zfu86V1DarUTnnkpEq5HjNNfDuuyGpuOTdcw+sWJGXzVoQr7P9nzG3OeeKwTnnhJ8T\nJyYZhUs3fjz07g0HH5x0JBnV10eyvaT9gM0l7SNp3+h2GNAuZxE653KrR4+wVokXcswPb70FTz6Z\nNwUaM6mvj+Ro4BzCyoY3pW1fAVyRxZicc0kbPhyGDIF//CMkFZecPCvQmEl9fSSTgEmSTjGze3IY\nk3MuaSedBFtvHUYKeSJJztq1MGkSDBoEXbokHU2dGlwh0czukXQssDthKdzU9rHZDMw5l6BUIcc7\n7gjDgbfeOumISlOqQONvf5t0JPWKU7Tx98AZwEWACOuo+1AO54pdqpDjtGlJR1K6Kiuhc+e8KtCY\nSZxRWweZ2VnAZ2Z2DdAf2DG7YTnnErf33qGYoxdyTMZHH20o0LjZZklHU684ieTL6OcqSV2ANUCv\n7IXknMsbI0bAiy+Gm8utKVPLwH4FAAAYGklEQVRCH0meFWjMJO6a7VsBNwDzgMWA19pyrhR4Icdk\npAo0Hngg9O2bdDQNilO08adm9nk0cqsHsJuZXRXn4JIGSnpD0iJJl2d4vrukxyS9KGm+pEHR9gpJ\nL6Xd1kvaO3puP0mvRMf8tZSnA6udKwZbbw3f/S5MneqFHHPpuedg4cK8ncleW30TEr9b+wYcCwyI\n7tdLUkvgNuAYoC8wRFLt1DoamGFm+wCDgd8BmNlUM9vbzPYGzgQWm1lqkYTbCTXAeke3gZtwvs65\nTZUq5HjffUlHUjryuEBjJvUN/z0++tkZOAj4R/T4cOBxQkn5+vQDFpnZOwCS7gJOJCzXm2JAqgLZ\nlsCyDMcZAkyPjrED0NHM5kSP7wROAh5sIBbnXGMdfjj07BmaWgYPTjqa4rdyJdx1F5x+OnTokHQ0\nsdR5RWJm55rZuYQP+75mdoqZnUKYTxJHV0LZ+ZTqaFu6McAwSdXAbMIQ49rOIEok0eurGzgmAJJG\nSqqSVFVTUxMzZOfcRlKFHB99FBYvTjqa4jdzZl4XaMwkTmd7TzN7P+3xh8AuMV6Xqe/Caj0eAkw0\ns26EdeAnS/pvTJIOAFaZ2aubcMyw0WycmZWbWXlZWVmMcJ1zdfJCjrkzfjzssgt861tJRxJbnETy\nuKSHJZ0j6WzgAeCxGK+r5uvzTbqxcdPVCGAGQNRc1RbolPb8YDZcjaSO2a2BYzrnmlv37nDkkV7I\nMdvefBOeeiqvCzRmEmfU1ijgD8A3gb2BcWaWqQmqtheA3pJ6SWpNSAqzau2zBBgAIKkPIZHURI9b\nEGbR/3eocXRltELSgdForbMAX8rNuVwYPhyWLAlNXC47JkyAli3zukBjJg3W2gIws9R67bGZ2VpJ\no4CHgZZApZktkDQWqDKzWcClwB2SLiE0UZ1jZqmmqkOB6lRnfZrzgYnA5oROdu9ody4XTjoprOte\nWRmuTlzzSi/QuMMOSUezSepMJJKeNrODJa3g6/0QAszMGlzv0cxmEzrR07ddnXZ/IZCxIdDMHgcO\nzLC9Ctijofd2zjWzNm1CIcdx4+DTT0NScc3noYfg/fcLYiZ7bfWN2jo4+tnBzDqm3TrESSLOuSLk\nhRyzJ1Wg8dhjk45kk9U3IXGb+m65DNI5lye++U3Ybz8v5NjcPvwQ/vrX0DeS5wUaM6mvj2QuoUmr\nriG3O2UlIudcfhs+HC68MBRy3GefpKMpDgVUoDGT+pq2epnZTtHP2jdPIs6VqqFDw8JXflXSPFIF\nGvv3hz59ko6mUeLMI0HS1pL6STo0dct2YM65PLXVVhsKOX75ZcP7u/o9+yy89lpBzWSvLc4Kif8D\nPEkYxntN9HNMdsNyzuW1ESPg88+9kGNzqKyE9u1Dba0CFeeK5HvA/sC7ZnY4sA/RpEHnXIk67DDo\n1cubt5rqiy8KrkBjJnESyWozWw0gqY2ZvQ7smt2wnHN5zQs5No+ZM0MyKeBmLYiXSKqjFRLvA/4m\n6S94fSvn3Nlnh3pQEyYkHUnhGj8edt0VDjoo6UiaJE6trZOjFRLHAFcB4wlrgDjnSln37nDUUSGR\nrFuXdDSF58034emnC65AYyZxOttvlXQQgJk9YWazzOyr7IfmnMt7w4fDe+95IcfGqKwsyAKNmcRp\n2poHjI7WSL9BUnm2g3LOFYgTTwwjjk4+OfSb9OwZhgW7+qUKNB57LGy/fdLRNFmcpq1JZjaIsHTu\nm8AvJb2V9cicc/lv5sxQe2vVqjCx7t13YeRITyYNefBB+OCDgp3JXlusCYmRnYHdgJ7A61mJxjlX\nWK68Mny7TrdqVdju6lZZCdttF0rGF4E4fSSpK5CxwAJgPzM7PuuROefy35Ilm7bdhQKN999fsAUa\nM4mzsNW/gP5m9nG2g3HOFZju3UNzVqbtLrPJkwu6QGMmcfpIfp9KIpLGZD0i51zhuPZaaNfu69ta\ntgzb3cZSBRoPOgh22y3paJrNpvSRAJyQlSicc4UptWJijx5hLsRWW4U5JR98kHRk+WnOHHj99YKf\nyV7bpiaSwp4145xrfhUVoUzK+vVhCd7vfhcuuwyeeirpyPJPERRozGRTE8l+WYnCOVccpPBhudNO\ncMYZfmWS7osv4E9/Cr+XLbZIOppmFWfU1vWSOkrajFBr62NJw3IQm3OuEG25JdxzTygzP2TIxsOD\nS9XddxdFgcZM4lyRHGVmy4HjgGpgF+D/xTm4pIGS3ohmxV+e4fnukh6T9KKk+ZIGpT23l6Q5khZI\nekVS22j749ExX4punWOdqXMud/bcE37/e3j8cbjqqqSjyQ+pAo39+ycdSbOLM/w3NdB5EDDdzD5V\njAJjkloCtwFHEhLQC5JmmdnCtN1GAzPM7HZJfYHZQE9JrYApwJlm9rKkbYE1aa+rMLOqGLE755Jy\n1lnwzDNw3XXhw/OEEh6r88Yb4Xdx/fUFX6AxkzhXJH+V9DpQDjwqqQxYHeN1/YBFZvZOVOTxLuDE\nWvsY0DG6vyUbytMfBcw3s5cBzOwTM/Pyos4VmltvhX33DUnlnXeSjiY5qQKNZ56ZdCRZEWceyeVA\nf6DczNYAK9k4IWTSFXgv7XF1tC3dGGCYpGrC1chF0fZdAJP0sKR5kn5U63UTomatq1TH5ZGkkZKq\nJFXV1PiCjs4lom3bUI+rRQs49VRYHec7aJFZsyYUaDzuuKIo0JhJnM7204C1ZrZO0mhCk1OXGMfO\n9AFvtR4PASaaWTdC09lkSS0ITW4HAxXRz5MlDYheU2FmewKHRLeMKd7MxplZuZmVl5WVxQjXOZcV\nvXqF2dwvvggXXdTw/sXmwQdDWZQimsleW5ymravMbIWkg4GjgUnA7TFeVw3smPa4GxuvrDgCmAFg\nZnOAtkCn6LVPmNnHZraKcLWyb7Tf0ujnCmAaoQnNOZfPjj0WrrgC/vhHmDgx6Whyq7IyXIkUSYHG\nTOIkklTfxLHA7Wb2F6B1jNe9APSW1EtSa2AwMKvWPkuAAQCS+hASSQ3wMLCXpHZRx/u3gYWSWknq\nFO2/GWEk2asxYnHOJW3sWDjiCDj/fHj55aSjyY0PPthQoLFVnLFNhSlOIlkq6Q/A6cBsSW3ivM7M\n1gKjCEnhNcLorAWSxkpKDd+4FDhP0svAdOAcCz4DbiIko5eAeWb2ANAGeFjS/Gj7UuCOTThf51xS\nWraE6dNhm23glFPCPJNiN3lyKBlTxM1aADKr3W1RawepHTAQeMXM3pK0A7CnmT2SiwCbQ3l5uVVV\n+Whh5/LCM8/AYYeFzud77y3K4bBAKNDYpw906hTWZi9AkuaaWYOr4sa5slgFvA0cLWkU0LmQkohz\nLs9861thPsV998GvfpV0NNnzz3+G+SNFOJO9tjijtr4HTAU6R7cpkkpw6IVzrtl8//thOPCPfwxP\nPpl0NNlRWRlqap12WtKRZF2cpq35hIWtVkaP2wNzzGyvHMTXLLxpy7k8tHw57L9/+DlvHuywQ9IR\nNZ8VK8L5DB4cRqoVqGZr2iLMB0mfVb4OLyfvnGuqjh1Dccfly8MHbjEVd7z7bli5siSatSBeIpkA\nPCdpTLRC4rPA+KxG5ZwrDXvsAX/4Q2jeuvLKpKNpPuPHhxUQDzww6UhyosGBzWZ2k6THCTPMBZxr\nZi9mOzDnXIkYNmxDQcODDoIT41RgymOvvx462m+4oXhHpNVSbyKJypXMN7M9gHm5Cck5V3JuuQWq\nquDss2HuXPjGN5KOqPEqK8PkwyIt0JhJvU1bZrYeeFlS9xzF45wrRW3ahH6FFi3CZMUvv0w6osZZ\nswbuvDPMkdluu6SjyZk4fSQ7AAskPSppVuqW7cCccyWmZ0+YMiWUTxk1KuloGmf27KIv0JhJnOIv\n12Q9Cuecg1DYcPRo+NnPQn9JoY16ShVoPOaYpCPJqToTiaSdge3M7Ila2w8l1LhyzrnmN2YMPPss\nXHhhWBRrn32Sjiie99+HBx6AH/6wqAs0ZlJf09YtwIoM21dFzznnXPNr2RKmTQs1qk49tXCKO5ZI\ngcZM6kskPc1sfu2N0VrpPbMWkXPOlZWFzvclS8JIrvXrk46ofmZh7sghh8AuuyQdTc7Vl0ja1vPc\n5s0diHPOfU3//nDjjTBrVpiTkc+eeQbefLMkr0ag/kTygqTzam+UNAKYm72QnHMuctFFcPrpYXXF\nxx9POpq6lVCBxkzq6xH6PvBnSRVsSBzlhNURT852YM45hxSKHs6fH+pxvfhi/hV3XLECZsyAIUOg\nffuko0lEnVckZvahmR1EGP67OLpdY2b9zeyD3ITnnCt5HTrAzJnhA/uMM8Kkv3wyY0ZJFWjMJM7C\nVo+Z2W+i2z9yEZRzzn3N7rvDHXfAU0+FZq58Mn58WAnxgAOSjiQxcWa2O+dc8oYOhQsuCKsq3ntv\n0tEEr70Gc+aEq5ESKdCYiScS51zhuOkm6NcPzj0X3nor6WhKskBjJllNJJIGSnpD0iJJl2d4vruk\nxyS9KGm+pEFpz+0laY6kBZJekdQ22r5f9HiRpF9LJfw1wLlS06ZN6JNo1SoUd1y1KrlYUgUajz8e\nOndOLo48kLVEIqklcBtwDNAXGCKpb63dRgMzzGwfYDDwu+i1rYApwP+Z2e7AYUCqh+12YCTQO7oN\nzNY5OOfyUI8eMHUqvPpqaOpqYLnwrHngAfjoo5KdO5Ium1ck/YBFZvaOmX0F3AXUXrHGgI7R/S2B\nZdH9owjroLwMYGafmNk6STsAHc1sjoXF5u8ETsriOTjn8tHAgXDVVTBpUnJroldWhqHIA/27bDYT\nSVfgvbTH1dG2dGOAYZKqgdnARdH2XQCT9LCkeZJ+lHbM6gaO6ZwrBVdfDUcdFSYtzsvxunvvvx9K\nxp99dskVaMwkm4kkU99F7WvQIcBEM+sGDAImR6sytiIs7VsR/TxZ0oCYxwxvLo2UVCWpqqamprHn\n4JzLVy1bhiausrLQX/LZZ7l77zvvLNkCjZlkM5FUAzumPe7GhqarlBHADAAzm0Oo79Upeu0TZvax\nma0iXK3sG23v1sAxiY43zszKzay8rKysGU7HOZd3OnUKxR2XLoWzzspNcUez0Kx16KHQu3f2368A\nZDORvAD0ltRLUmtCZ3rtlRWXAAMAJPUhJJIa4GFgL0ntoo73bwMLzex9YIWkA6PRWmcBf8niOTjn\n8t2BB4ZhwfffD7/8Zfbf7+mnS7pAYyZZSyRmthYYRUgKrxFGZy2QNFbSCdFulwLnSXoZmA6cY8Fn\nwE2EZPQSMM/MHohecz7wR2AR8DbwYLbOwTlXIC68MNTiGj0aHnssu+9VWRnKtpx6anbfp4DIkho6\nl0Pl5eVWVVWVdBjOuWz64oswWfGTT0Lne9csjMNZvjyM1KqogHHjmv/4eUbSXDMrb2g/n9nunCsO\nW2wB99wTCihmq7jjjBlhEmQJF2jMxBOJc6549OkT5pU88wxcdlnzH3/8eOjbN1z5uP/yROKcKy6D\nB8OoUXDzzaH8fHNZuBCefbbkCzRm4onEOVd8brwxlHUfPjyMsGoOqQKNw4Y1z/GKiCcS51zxad06\n9Ge0bh0mK65c2bTjpQo0nnBCyRdozMQTiXOuOHXvDtOmwYIFcP75TSvueP/9UFPjc0fq4InEOVe8\njjoKfvITmDy5acN1KyuhSxc4+ujmi62IeCJxzhW3q64KCeDii6Ex88mWLfMCjQ3wROKcK24tWsCU\nKbD99mE2+qefbtrr77wz1PDyZq06eSJxzhW/VHHHZcvCsrhxizumCjR++9uw887ZjbGAeSJxzpWG\nfv3glltCM9UvfhHvNU89FdaG96uRenkicc6VjvPPh6FDw6JYf/97w/t7gcZYPJE450qHFEZv7bYb\nDBkC1dV177t8eWgOGzIE2rXLXYwFyBOJc660tG8fijuuXg2nnw5ffZV5vz/9yQs0xuSJxDlXenbb\nLRRgnDMHfvSjzPuMHw+77w7775/b2AqQJxLnXGk6/fQwt+TWW0M5lXQLFsBzz3mBxpg8kTjnStcN\nN0D//iFhvP76hu2VlbDZZl6gMSZPJM650pUq7ti2bRiZtXJl6DOZPDkUaCwrSzrCguCJxDlX2rp1\nC8UdFy4Mtbm6dg0FGp98EqZOTTq6guCJxDnnjjwylJv/5z/h44/DtpoaGDnSk0kMnkiccw7g+ec3\n3rZqFVx5Ze5jKTBZTSSSBkp6Q9IiSZdneL67pMckvShpvqRB0faekr6U9FJ0+33aax6Pjpl6zleZ\ncc413XvvZd6+ZElu4yhAWauJLKklcBtwJFANvCBplpktTNttNDDDzG6X1BeYDfSMnnvbzPau4/AV\nZtaIetDOOVeH7t3h3Xczb3f1yuYVST9gkZm9Y2ZfAXcBJ9bax4CO0f0tgWVZjMc55+p27bUbl0Jp\n1y5sd/XKZiLpCqRfK1ZH29KNAYZJqiZcjVyU9lyvqMnrCUmH1HrdhKhZ6yop82whSSMlVUmqqqmp\nadqZOOeKX0VFqMPVo0eYhNijR3hcUZF0ZHkvm4kk0wd87UWThwATzawbMAiYLKkF8D7Q3cz2AX4A\nTJOUunKpMLM9gUOi25mZ3tzMxplZuZmVl/lYcOdcHBUVsHhxWK9k8WJPIjFlM5FUAzumPe7Gxk1X\nI4AZAGY2B2gLdDKz/5jZJ9H2ucDbwC7R46XRzxXANEITmnPOuYRkM5G8APSW1EtSa2AwMKvWPkuA\nAQCS+hASSY2ksqizHkk7Ab2BdyS1ktQp2r4ZcBzwahbPwTnnXAOyNmrLzNZKGgU8DLQEKs1sgaSx\nQJWZzQIuBe6QdAmh2escMzNJhwJjJa0F1gH/Z2afSmoPPBwlkZbA34E7snUOzjnnGiaz2t0Wxae8\nvNyqqny0sHPObQpJc82svKH9fGa7c865JimJKxJJNUCGmUZ5oxPwcdJBNBM/l/xTLOcBfi651sPM\nGhz2WhKJJN9Jqopz+VgI/FzyT7GcB/i55Ctv2nLOOdcknkicc841iSeS/DAu6QCakZ9L/imW8wA/\nl7zkfSTOOeeaxK9InHPONYknEuecc03iiSQPSGoZlcy/P+lYmkLSVpJmSnpd0muS+icdU2NIukTS\nAkmvSpouqW3SMcUlqVLSR5JeTdu2jaS/SXor+rl1kjHGVce53BD9/5ov6c+StkoyxrgynUvacz+U\nZKk6goXIE0l++B7wWtJBNINbgYfMbDfgmxTgOUnqClwMlJvZHoSaboOTjWqTTAQG1tp2OfComfUG\nHo0eF4KJbHwufwP2MLO9gDeBH+c6qEaayMbngqQdCavIFvR6vp5IEiapG3As8MekY2mKaL2YQ4Hx\nAGb2lZl9nmxUjdYK2FxSK6AdBbRyp5k9CXxaa/OJwKTo/iTgpJwG1UiZzsXMHjGztdHDZwnLU+S9\nOv5dAG4GfsTGazUVFE8kybuF8B9pfdKBNNFOQA1h9coXJf0xqtZcUKL1bn5F+Ib4PvBvM3sk2aia\nbDszex8g+tk54Xiay3DgwaSDaCxJJwBLzezlpGNpKk8kCZJ0HPBRtHhXoWsF7AvcHq1suZLCaUL5\nr6j/4ESgF9AFaC9pWLJRudokXQmsBaYmHUtjSGoHXAlcnXQszcETSbK+BZwgaTFwF3CEpCnJhtRo\n1UC1mT0XPZ5JSCyF5jvAv8ysxszWAPcCByUcU1N9KGkHgOjnRwnH0ySSziYsaldhhTsR7huELysv\nR3//3YB5krZPNKpG8kSSIDP7sZl1M7OehA7df5hZQX77NbMPgPck7RptGgAsTDCkxloCHCipnSQR\nzqPgBg3UMgs4O7p/NvCXBGNpEkkDgcuAE8xsVdLxNJaZvWJmnc2sZ/T3Xw3sG/0dFRxPJK45XQRM\nlTQf2Bv4ecLxbLLoimomMA94hfA3UjClLCRNB+YAu0qqljQCuA44UtJbhBFC1yUZY1x1nMtvgQ7A\n3yS9JOn3iQYZUx3nUjS8RIpzzrkm8SsS55xzTeKJxDnnXJN4InHOOdcknkicc841iScS55xzTeKJ\nxLkESPoi7f6gqDJv9yRjcq6xWiUdgHOlTNIA4DfAUWZW0BVgXenyROJcQiQdAtwBDDKzt5OOx7nG\n8gmJziVA0hpgBXCYmc1POh7nmsL7SJxLxhrgn0BRlcpwpckTiXPJWA+cDuwv6Yqkg3GuKbyPxLmE\nmNmqaE2apyR9aGbjk47JucbwROJcgszs06g0+pOSPjazgi3x7kqXd7Y755xrEu8jcc451ySeSJxz\nzjWJJxLnnHNN4onEOedck3gicc451ySeSJxzzjWJJxLnnHNN8v8BVRw8q4RcAYAAAAAASUVORK5C\nYII=\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0xa4ffe80>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "# Plot the cross validation\n",
    "plt.plot(K_classes, K_accuracy, 'ro-')\n",
    "plt.title('Cross-validation on k')\n",
    "plt.xlabel('K')\n",
    "plt.ylabel('Cross-validation accuracy')\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "可见，K值取3的时候，验证集的准确率最高。此例中，由于总体样本数据量不够多，所以验证结果并不明显。但是使用k-fold交叉验证来选择最佳K值是最常用的方法之一。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 对测试集进行预测"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "选择完合适的K值之后，就可以对测试集进行预测分析了。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 194,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "测试集预测准确率：1.000000\n"
     ]
    }
   ],
   "source": [
    "KNN.train(X_train, y_train)\n",
    "y_pred = KNN.predict(X_test, k=6)\n",
    "accuracy = np.mean(y_pred == y_test)\n",
    "print('测试集预测准确率：%f' % accuracy)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "最终结果显示，测试集预测准确率为100%。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "最后，我们把预测结果绘图表示。仍然只选择sepal length和petal length两个特征，在二维平面上作图。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 204,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXwAAAEKCAYAAAARnO4WAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMS4wLCBo\ndHRwOi8vbWF0cGxvdGxpYi5vcmcvpW3flQAAIABJREFUeJzt3Xt81OWV+PHPCfcoKIIiErn5KiLk\nyk0uKiAICuhagZdUvIClFBHarotF6g+lstTuYhdLV2vRFamyNTWo1W5bKRQQFYWAAUQuIgaEhBBi\niQFCAuT8/pjJkMtk8p3kO5OZzHm/XvNK5jvfy5mv+OSZM89zHlFVjDHGNH5xDR2AMcaY8LAG3xhj\nYoQ1+MYYEyOswTfGmBhhDb4xxsQIa/CNMSZGWINvjDExwhp8Y4yJEdbgG2NMjGja0AFU1L59e+3a\ntWtDh2GMMVFj69atx1X1cif7RlSD37VrVzIzMxs6DGOMiRoictDpvpbSMcaYGGENvjHGxAhr8I0x\nJkaErMEXkWtFJKvC41sR+UmormeMMSawkH1pq6p7gVQAEWkCHAHeCtX1jDHGBBaulM4I4EtVdfxt\nsjHGGHeFq8GfBPwhTNcyxhjjR8jH4YtIc+AOYF4Nr08HpgN07tw51OEYY0yDavN0G4pKi6ptb928\nNd/O+zak1w5HD/82YJuq5vl7UVWXqWo/Ve13+eWOJosZY0zU8tfYB9rupnA0+N/D0jnGGNPgQtrg\ni0g8cAvwZiivY4wxpnYhzeGr6mmgXSivYYwxxhmbaWuMMTHCGnxjjAmj1s1bB7XdTdbgG2NMGDX2\nUTrGGGMigDX4xhgTIyJqxStjjIlmDTmL1gnr4RtjjEsaMj/vhDX4xhgTI6zBN8aYGGE5fGOMCSN9\nUhvs2tbDN8aYGGENvjHGuKQhZ9E6YSkdY4xxSSQMvQzEevjGGBMjrME3xpgYYSkdY0yjFumzX8PJ\nevjGmEYt0me/hpM1+MYYEyOswTfGmBhhDb4xxsQIa/CNMSZGWINvjGnUIn32azjZsExjTNRyMuQy\n1oZeBmI9fGNM1LIhl8EJaYMvIpeKSIaI7BGR3SIyKJTXM8YYU7NQp3R+DfxNVSeISHMgPsTXMybq\n2ExQEy4ha/BFpA1wEzAFQFVLgdJQXc+YaGVpCRMuoUzpdAfygeUi8qmIvCQiF4XwesYYYwIIZUqn\nKdAHmK2qn4jIr4HHgPkVdxKR6cB0gM6dO4cwHGOMGzZubMP589U/fTRp0pobb3Q3BWXpLneFsod/\nGDisqp94n2fg+QNQiaouU9V+qtrv8ssvD2E4xhg3+GvsA22vD0t3uStkDb6qHgW+FpFrvZtGAJ+H\n6nrGGGMCC/UondnASu8InQPA1BBfz5io07p56xrTFiYwu3fBCWmDr6pZQL9QXsOYaOdGLtqtXLfl\nzBs3m2lrTCPgVq472nLm0RZvQ7MG3xgTlCZN/KdLatpeH1b4zF1WPM0Y4/PnIXCRn1bh1LkLvzsZ\neulWasjSSO6yHr4xxsdfYx9oe00s1RKZrME3xpgYYSkdYxqYk/RHbfvE6vDEWH3fdWUNvjENzEn6\no7Z9YjXXHavvu64spWOMMTHCGnxjjOtsOGVkspSOMY1AOGfI2jqy0ct6+MY0AuEcBmlDLqOXNfjG\nNDAn6Y9wpUjCOYvWhJ+ldIwJISeLhThJf4QrReL2AiYmslgP35gQCudiIcbUxhp8Y4yJEZbSMSYK\nyM+lxtf0SQ3rjFOb3Rq9rME3phEI5zBIG3IZvSylY4wxMcIafGNCyIY5mkhiKR1jQihcwxydDP8M\n53lMZLIevjGNgFvDP20YaeNmDb4xxsQIS+mYRuX/1kiNa7KOHalAdKYt9Elt6BCCEo33OBZYD980\nKk7WZLW0RejZPY5MIe3hi0g2UAScB86par9QXs8YY0zNwpHSGa6qx8NwHRMDwlX3PZz15d1If5w6\n5//TzalzwcXSpEnrGmMx0c9y+CaqhKsWezhrvruR/hj3Yc2v6UjnsZT/gZk5E373O5gxA557zvnx\nJrKFOoevwGoR2Soi00N8LWOMC3JzYflyKCvz/Dx6tKEjMm4JdQ9/iKrmiMgVwN9FZI+qvl9xB+8f\ngukAnTt3DnE4JpK5kUZxktpwK20RzrRPbdwsaLZwoaexBzh/3vM82F6+pYYiU0gbfFXN8f48JiJv\nAQOA96vsswxYBtCvX7/oGntmXOVGGqV86GUgbg0LjKSl/tz6A1Peuy8t9TwvLfU8nz8frrzS+Xks\nNRSZQpbSEZGLRKR1+e/AKOCzUF3PGFN/FXv35cp7+cGy1FDkCWUOvwPwgYhsBzYD/6eqfwvh9UwM\nCNfaruG6DtQ8kibYETZueOedC737cqWl8Kc/BX8uf6kh07BCltJR1QNASqjOb2KTm7nx3Fy44Qb4\n8MPq6Qq3ruMkz+/WCBs3HD7sfN9A98+t1JBxl820NTFr4ULIzg5tzzOS8vxuC3T/3EwNGffU2uCL\nSAsRuUdEfiYiT5Q/whGciS3hTKO4kV92K14p9b9/TdsjQW33z83UkHGPk5TOn4BCYCtQEtpwTCwL\n51BGN4YelseblQX9+8PWrZCcHHwsZYs85wmUIok0td0/t1JDxl1OUjoJqnq3qv6nqv6q/BHyyIwJ\nkZryy3UdRXLvvXDuHNxzT/3iCkeKyQ1u379oed+NgZMG/yMRSQp5JMaEiZv55aws2LXL8/uuXbBj\nR91iiqYhjDZ0M3rV2OCLyE4R2QHcAGwTkb0isqPCdmOikpv55Xvvhcsuy+W1166hbduj1Xr5TvP8\nbg1hzM2Fa64J3HA62ScQG7oZxVTV7wPoEuhR03H1efTt21eNiRaffqoKqj/+8UO6dm2c/vjHMxVU\nt28P7jw5OaotW3rOVf5o1Uo1Nzf4mB56SDUuTnXmzPrtEw5uvu9YBmSqwza2xh6+qh5U1YPAv5f/\nXnFbqP8QGRPpynv3t922nLi4Mm69dbnfXn5t3EqROEmPRFIKxYZuhp+THH7vik9EpAnQNzThGBM9\ndu+G++5biIin1YqLO8/99y9k9+7gzuNWisRJeiSSUig2dDP8xPOJwM8LIvOAnwGtgNPlm4FSYJmq\nznM7mH79+mlmZqbbp406kVSFsSE4GaYXrqF8ga5TUpLLJ590p6zsjG9bXFwrrr/+AC1aXOn4PG7E\nk5sL3bvDmQuh0KoVHDhwYT8n+5joIyJb1eFqgoFSOk+ramtgsaq28T5aq2q7UDT25oLGPDvTCSfD\n9MI1lC/QdbKzF6JaOSehep6DB6vv7Fa8NZ3HSXrEUiimxh6+bweRPn42FwIHVdXV8k7Ww/eQn0uN\nr+mTjbuCdMVeaE29Tyf7hCOWjz5KoLT0SLXjmjfvxODBhx2fx414EhLgSPVQ6NTpwiQoJ/uY6ONK\nD7+C54GP8dSsf9H7++vAPhEZVecoTUwqKcnl44+voaTE/7eFkZSHru06gwcfZtgwrfao2Ni7GW+g\n8xw+XHGsy4VHxYa8fJ+cHM8fjtzc6vuUq+/QTROZnDT42UCaqvZT1b5AKp669iOB/wxhbKYRys5e\nyJkz2X7THk5mcLo9y7Mmbl0n0s4DkZUyM+HlpMHvqaq7yp+o6ud4/gAcCF1YpjEqKcklL285UMbR\no8ur9fIjKQ/t1nUi7TzRNnTTuMtJg79XRH4rIkO9j+fxpHNaAGdDHF9MCmfVyHCq+CWnvy83nQzT\nC9dQPreuU36eirNx63Oe+sYTSSkzE35OvrRtBczEU2JBgA/w5PXPAPGqetKtYOxL28YrmCGMjdHe\nvTPJzf0dV101gx496r64a33WiLWhm42Tq1/aqmqxeipkfldV71TVZ1T1tKqWudnYm8YtmCGMjU1t\nqSyn6ptqiaSUmWkYThZAGSIifxeRfSJyoPwRjuBM41FQ8A6qlXMSqqUcP974p1XWlspyqr6plkhK\nmZmG4SSlswf4VzwLoJwv366qBW4HYykdA54e8aef3kBa2oc1pnuKirLYurU/fftupXXrOqw6Qnhm\n67qVyrJUi6mJ2+PwC1X1r6p6TFULyh/1jNGYGgUaullu9+57gXPs3l33VUfCMfTQrVSWpVqMG5w0\n+OtEZLGIDBKRPuWPkEdmYpKTfHdRURanT3tGCp8+vYuiouCXZwjX0EO3UlmWajFucLKm7fXenxU/\nMihws/vhmFjnL99ddVSLp3df8fk9DBjwWaVttaVr3FjT1omqs27rykofGDc4GaUz3M/DGnvjuvLe\nfXmPWLW0Wi+/Yu++nL9efqB0Tbhm6xoTaZyM0ukgIv8jIn/1Pu8lIt8PfWgm1jjJd1ft3V/YfiGX\nX1u6xvLhJlY5yeG/ArwHXOV9vg/4idMLiEgTEflURP4cfHimsQlUlMtJvvv0af+ri1TcvnAhXHqp\nZ2ZrmzZHqzXkweTDw7FGrDHh4qTBb6+qfwTKALwlkc8HPqSSHwNBrgFkGqtAqRYn1ScHDTpMXFzL\nSsfFxbVi0CBP3d/y3v3ddy+kY8dsJk1aWK2X76SypJN4g9nHmEjgpME/JSLt8HxRi4gMxFMPv1Yi\nkgCMBV6qc4Sm0XBjZExtaZ/y3n3FdWb99fLditcKjZlo4qTBfwR4B7hGRD4Efg/Mdnj+Z4Gf4v10\n4I+ITBeRTBHJzM/Pd3haE43cKMpVW9rnnXc8vfuK68xOmrSwTsMXrdCYaWxqnWkLICJNgWvxFE/b\nq6q1VskUkXHAGFWdKSLDgDmqOi7QMTbTtmGEY8ZpMDNF6xNPOGe22uxXEwlcmWkrIneVP4A78DT4\nPYDbvdtqMwS4Q0Sy8ayQdbOIvOYkKBNe4chBBzMypj7xhHNmq432MdGmxh6+iCwPcJyq6oOOL2I9\n/IgVrvVhna6nWt94nK4z60a8tkasiQTB9PBrnGmrqlPdC8lEqnDNOHXaANY3nnDObLVG3UQbRzn8\ncLEefnhFWg460uIxJhq40sM3jV+gHHQoevnRFk9jc/bsWQ4fPsyZin9RTdRo2bIlCQkJNGvWrM7n\nsB5+DIu0HHSkxdPYfPXVV7Ru3Zp27dohIg0djgmCqlJQUEBRURHdunWr9JorPfzaRuKo6puOIjUR\nK9Ia0UiLp7E5c+YMXbt2tcY+CokI7dq1o75zlQKldG4P8JoC1uAbE2WssY9ebvy3s1E6JuKEYyKY\nMbHI0Ze2IjIW6A34qlap6lOhCsrEtooTr+zLWmPc46Qe/gvA3Xjq5wgwEegS4rhMjLJiZKbcK6+8\nQk5OTsivs379ej766KOgj8vMzORHP/pRCCIKHSfF0war6v3AP1X158Ag4OrQhmVilRUjiyArV0LX\nrhAX5/m5cmVYLx8JDf65c+dqPK5fv34sXbo0VGGFhJMGv9j787SIXAWcBboF2N+EUWNafCOYpQcb\n0/uOSCtXwvTpcPCgZ7GAgwc9z+vZ6J86dYqxY8eSkpJCYmIi6enpbN26laFDh9K3b19Gjx5Nbm4u\nGRkZZGZmMnnyZFJTUykuLmbt2rWkpaWRlJTEgw8+SElJCQCPPfYYvXr1Ijk5mTlz5gDw7rvvcv31\n15OWlsbIkSPJy8vzG092djYvvPACS5YsITU1lY0bNzJlyhQeeeQRhg8fzty5c9m8eTODBw8mLS2N\nwYMHs3fvXsDzh2LcOE+1mAULFvDggw8ybNgwunfvHrl/CFQ14AOYD1wKjAeOArnAwtqOq8ujb9++\naoLz0EOqcXGqM2c2dCT199BDqs2bV16WpHlz/++tMb3vcPn888+d79yli781Yjzb6yEjI0OnTZvm\ne37ixAkdNGiQHjt2TFVVX3/9dZ06daqqqg4dOlS3bNmiqqrFxcWakJCge/fuVVXV++67T5csWaIF\nBQXao0cPLSsrU1XVf/7zn6qq+s033/i2vfjii/rII4/UGNOTTz6pixcv9j1/4IEHdOzYsXru3DlV\nVS0sLNSzZ8+qqurf//53veuuu1RVdd26dTp27FjfOQYNGqRnzpzR/Px8veyyy7S0tLQ+t8ovf/8N\ngUx12MY66eH/p6qeUNVVeHL3PYF/D82fHxOMxpbvdrr0YGN73xHp0KHgtjuUlJTEmjVrmDt3Lhs3\nbuTrr7/ms88+45ZbbiE1NZV///d/57CfCRl79+6lW7du9OjRA4AHHniA999/nzZt2tCyZUumTZvG\nm2++SXx8PACHDx9m9OjRJCUlsXjxYnbt2lXtnIFMnDiRJk2aAFBYWMjEiRNJTEzkX//1X2s819ix\nY2nRogXt27fniiuuqPFTRUNy0uBvKv9FVUtUtbDiNtNwGlu+2+nSg43tfUekzp2D2+5Qjx492Lp1\nK0lJScybN49Vq1bRu3dvsrKyyMrKYufOnaxevbracVpDRYCmTZuyefNmxo8fz9tvv82tt94KwOzZ\ns5k1axY7d+7kd7/7XdDlJC666CLf7/Pnz2f48OF89tlnvPvuuzWeq0WLFr7fmzRpEjD/31AC1cO/\nUkT6Aq1EJE1E+ngfw4D4sEVo/Aom392YxOr7DrtFiyC+yv/m8fGe7fWQk5NDfHw89957L3PmzOGT\nTz4hPz+fTZs8fcizZ8/6etCtW7emqKgIgJ49e5Kdnc3+/fsBePXVVxk6dCgnT56ksLCQMWPG8Oyz\nz5KVlQV4euWdOnUCYMWKFQFjqngdfyqe65VXXqn7m48AgXr4o4FngATgv4BfeR//Cvws9KGZQGJ1\n8Y1Yfd9hN3kyLFsGXbqAiOfnsmWe7fWwc+dOBgwYQGpqKosWLeKpp54iIyODuXPnkpKSQmpqqm/E\nzJQpU5gxYwapqamoKsuXL2fixIkkJSURFxfHjBkzKCoqYty4cSQnJzN06FCWLFkCeL5EnThxIjfe\neCPt27cPGNPtt9/OW2+95fvStqqf/vSnzJs3jyFDhnD+/Pl6vf+GVmvxNBEZ783fh5wVT6ss0IzT\n8kJjl12Wy9KlNzB79of8859XNvpCY1Zgre52797Ndddd19BhmHrw99/QlSUOK/hQRP5HRP7qPXkv\nEfl+8KGaYAVa6q883/3RRwvp1Cmbjz9e6Dff3dg4zfMbY6pz0uAvB94DrvI+3wf8JGQRGcDZSJSS\nklzy8pYDZRw9upySEktkG1OT5cuXk5qaWunx8MMPN3RYYeWklk57Vf2jiMwDUNVzIhLdiawo4GSp\nv4oLdpcv1N2jhxWfMcafqVOnMnVqbNeEdNLDPyUi7fCUREZEBgKFIY0qxjkZiVLeu1f17KRaar18\nY0xAThr8R4B3gGtE5EPg93gKqZkQcTISpWLvvlx5L98YY/yptcFX1W3AUGAw8EOgt6ruCHVgsczJ\njNOCgnd8vftyqqUcP15lWqoxxnjVmsMXkZbATOAGPGmdjSLygqraSsgh4mTEyeDBNizFGBMcJymd\n3+NZ/OQ3wH8DvYBXQxmUMcY49cQTT7BmzZqgj6tY7bI+srKy+Mtf/hL0cTk5OUyYMKHe1w+Gk1E6\n16pqSoXn60Rke6gCMsZEjkhZbtJX7TGueh/1qafCs/jeuXPnaNq0epOZlZVFZmYmY8aMcXwMwFVX\nXUVGRobrcQbipIf/qXdkDgAicj3wYW0HiUhLEdksIttFZJeI/Lw+gZq6KSnJ5eOPr7HRO6ZOAk3+\nq4u5c+fy/PPP+54vWLCAX/3qVyxevJj+/fuTnJzMk08+CXhq1V933XXMnDmTPn368PXXXzNlyhQS\nExNJSkrylVGYMmWKr+HcsmULgwcPJiUlhQEDBlBUVMSZM2eYOnUqSUlJpKWlsW7dumpxffPNN9x5\n550kJyczcOBAduzY4Ytv+vTpjBo1ivvvv7/acaWlpTzxxBOkp6eTmppKenp6tWOys7O58cYb6dOn\nD3369PGVjsjOziYxMRHw1Oi56667uPXWW/nOd77DT3/6U3dueFW11U8GdgNlQLb3UQbsAnYCOwIc\nJ8DF3t+bAZ8AAwNdy+rhu2/Pnod03bo43bvXCsfHuqDq4atqTo5qy5aeucytWqnm5tY/hm3btulN\nN93ke37dddfpihUr9Ac/+IGWlZXp+fPndezYsbphwwb96quvVER006ZNqqqamZmpI0eO9B1bXvv+\ngQce0DfeeENLSkq0W7duunnzZlW9UMf+mWee0SlTpqiq6u7du/Xqq6/W4uLiSvXsZ82apQsWLFBV\n1bVr12pKSoqqeurc9+nTR0+fPl3je1q+fLk+/PDDvudVjzl16pQWFxerquq+ffu0vJ376quvtHfv\n3r5zdOvWTU+cOKHFxcXauXNnPXToULVr1bcevpOUzq11/EOiwEnv02beR+DCPcZVVWfidukynxYt\nGvBzuYkqTib/BSstLY1jx46Rk5NDfn4+bdu2ZceOHaxevZq0tDQATp48yRdffEHnzp3p0qULAwd6\nEgzdu3fnwIEDzJ49m7FjxzJq1KhK5967dy8dO3akf//+ALRp0waADz74gNmzPSPJe/bsSZcuXdi3\nb1+lYz/44ANWrfKUDLv55pspKCigsNAz3eiOO+6gVatWQb3PisecPXuWWbNmkZWVRZMmTapdu9yI\nESO45JJLAOjVqxcHDx7k6qvdXU3WybDMg4EegY4VkSYikgUcA/6uqp/42We6iGSKSGZ+fn7d34mp\nxt9MXGOcCGUZ6gkTJpCRkUF6ejqTJk1CVZk3b56vJv7+/fv5/vc95boq1qVv27Yt27dvZ9iwYTz3\n3HNMmzat0nlVFRGpdj2tpUBkTfuUn6tiDE5VPGbJkiV06NCB7du3k5mZSWnVMdde4ain7ySHX2eq\nel5VU/GUWB4gIol+9lmmqv1Utd/ll18eynBiis3ENfURyjLUkyZN4vXXXycjI4MJEyYwevRoXn75\nZU6e9CQEjhw5wrFjx6odd/z4ccrKyhg/fjwLFy5k27ZtlV7v2bMnOTk5bNmyBYCioiLOnTvHTTfd\nxErvWrz79u3j0KFDXHvttZWOrbjP+vXrad++ve8TQm2c1NPv2LEjcXFxvPrqqw1aYtlJSqfeVPWE\niKzHkx76LBzXjHWBZuJavR1Tm0CT/+qb1unduzdFRUV06tSJjh070rFjR3bv3s2gQYMAuPjii3nt\ntdd8SwyWO3LkCFOnTqXM+5fo6aefrvR68+bNSU9PZ/bs2RQXF9OqVSvWrFnDzJkzmTFjBklJSTRt\n2pRXXnmlUm8aPF/OTp06leTkZOLj42tdNKWi4cOH88tf/pLU1FTmzZtX7fWZM2cyfvx43njjDYYP\nH16nTwxuqbUefp1PLHI5cNbb2LcCVgP/oap/rukYq4fvno8+SqC0tHrh+ObNO9mkrRhl9fCjX33r\n4Yeyh98RWCEiTfCkjv4YqLE37rJG3RhTVcgafPXU20kL1fmNMSac3nvvPebOnVtpW7du3Xjrrbca\nKKLghSWHb4wx0W706NGMHj26ocOol5CO0jHGGBM5rME3xpgYYQ2+McbECGvwjTENqq5lgqdNm8bn\nn38ecJ8XXniB3//+93UNrZITJ05UKvwWjDFjxnDixAlX4qiPkI3Drwsbh29M6ETbOPxApYUbQnZ2\nNuPGjeOzz6rPHT1//ny1iWKhUN9x+NbDN8ZU0+bpNsjPpdqjzdPOyg3UpKbyyBXLBE+cOJHbb7+d\nUaNGUVZWxsyZM+nduzfjxo1jzJgxvlLIw4YNo7yDePHFF/P444+TkpLCwIEDycvL853/mWeeAWD/\n/v2MHDmSlJQU+vTpw5dffsnJkycZMWIEffr0ISkpiT/9qeYlQh977DG+/PJLUlNTefTRR1m/fj3D\nhw/nnnvuISkpCYA777yTvn370rt3b5YtW+Y7tmvXrhw/ftxX8vkHP/gBvXv3ZtSoURQXF9frngbD\nGnxjTDVFpf5rw9S03alJkyaRnp7ue/7HP/7RV92y3KZNm1ixYgX/+Mc/ePPNN8nOzmbnzp289NJL\nbNq0ye95T506xcCBA9m+fTs33XQTL774YrV9Jk+ezMMPP8z27dv56KOP6NixIy1btuStt95i27Zt\nrFu3jn/7t3+rsdjaL3/5S6655hqysrJYvHgxAJs3b2bRokW+1NLLL7/M1q1byczMZOnSpRQUFFQ7\nzxdffMHDDz/Mrl27uPTSS31VOsMhcj4vGWMaPX/lkTt37lxpn1tuuYXLLrsM8JQtnjhxInFxcVx5\n5ZUMHz7c73mbN2/uW66wb9++/P3vf6/0elFREUeOHOG73/0uAC1btgQ8pYt/9rOf8f777xMXF8eR\nI0fIy8vjSofLew0YMIBu3br5ni9dutQ3Eevrr7/miy++oF27dpWO6datG6mpqb5Ys7OzHV3LDdbg\nG2PCqrw88tGjR5k0aVK11ysWF3P6HWOzZs185Yz9lRau6TwrV64kPz+frVu30qxZM7p27cqZM2ec\nvpVKsa5fv541a9awadMm4uPjGTZsmN9zVS2DbCkdY0yjVbU8ciA33HADq1atoqysjLy8PNavX1+n\na7Zp04aEhATefvttAEpKSjh9+jSFhYVcccUVNGvWjHXr1nHwYM1LfDgpg9y2bVvi4+PZs2cPH3/8\ncZ1iDSVr8I0xYVW1PHIg48ePJyEhgcTERH74wx9y/fXX+1aFCtarr77K0qVLSU5OZvDgwRw9epTJ\nkyeTmZlJv379WLlyJT179qzx+Hbt2jFkyBASExN59NFHq71+6623cu7cOZKTk5k/f75vpa5IYsMy\njYkRwQzLbPN0G79f0LZu3ppv533rdmgBnTx5kosvvpiCggIGDBjAhx9+6DjH3thEcnlkY0yUCnej\nHsi4ceM4ceIEpaWlzJ8/P2YbezdYg2+MiWh1zdvXVUFBASNGjKi2fe3atdVG3EQba/CNMaaCdu3a\nkZWV1dBhhIR9aWuMMTHCGnxjjIkR1uAbY0yMsAbfGBORXnnlFXJyckJ+nfXr1/PRRx/V6djs7Gz+\n93//1+WIQscafGNMRLIG333W4Btj/MrLW8mmTV1Zvz6OTZu6kpe3st7nPHXqFGPHjiUlJYXExETS\n09PZunUrQ4cOpW/fvowePZrc3FwyMjLIzMxk8uTJpKamUlxczNq1a0lLSyMpKYkHH3yQkpISwFO2\nuFevXiQnJzNnzhwA3n33Xa6//nrS0tIYOXKkr1xyVdnZ2bzwwgssWbKE1NRUNm7cSH5+PuPHj6d/\n//7079+fDz/8EIANGzaQmpraPQ+dAAAZwElEQVRKamoqaWlpFBUV8dhjj7Fx40ZSU1NZsmRJve9P\nyKlqxDz69u2rxpjQ+Pzzzx3ve/Toa7phQ7yuW4fvsWFDvB49+lq9YsjIyNBp06b5np84cUIHDRqk\nx44dU1XV119/XadOnaqqqkOHDtUtW7aoqmpxcbEmJCTo3r17VVX1vvvu0yVLlmhBQYH26NFDy8rK\nVFX1n//8p6qqfvPNN75tL774oj7yyCM1xvTkk0/q4sWLfc+/973v6caNG1VV9eDBg9qzZ09VVR03\nbpx+8MEHqqpaVFSkZ8+e1XXr1unYsWPrdU+C4e+/IZCpDttYG4dvjKnmwIHHKSs7XWlbWdlpDhx4\nnA4dJtf5vElJScyZM4e5c+cybtw42rZty2effcYtt9wCeFaO8ldfZ+/evXTr1o0ePXoA8MADD/Dc\nc88xa9YsWrZsybRp0xg7dqyvRPLhw4e5++67yc3NpbS0tFIJ49qsWbOm0tKJ3377LUVFRQwZMoRH\nHnmEyZMnc9ddd5GQkFDn+9BQQpbSEZGrRWSdiOwWkV0i8uNQXcsY466SkkNBbXeqR48ebN26laSk\nJObNm8eqVavo3bs3WVlZZGVlsXPnTlavXl3tOK2h5lfTpk3ZvHkz48eP5+233+bWW28FYPbs2cya\nNYudO3fyu9/9LqiSx2VlZWzatMkX05EjR2jdujWPPfYYL730EsXFxQwcOJA9e/bU7SY0oFDm8M8B\n/6aq1wEDgYdFpFcIr2eMcUmLFp2D2u5UTk4O8fHx3HvvvcyZM4dPPvmE/Px830pWZ8+eZdeuXUDl\ncsQ9e/YkOzub/fv3A57Kl0OHDuXkyZMUFhYyZswYnn32Wd8M2cLCQjp16gTAihUrAsZUtezxqFGj\n+O///m/f8/JzfvnllyQlJTF37lz69evHnj17ai2ZHGlC1uCraq6qbvP+XgTsBjqF6nrGGPd0776I\nuLj4Stvi4uLp3n1Rvc67c+dOBgwYQGpqKosWLeKpp54iIyODuXPnkpKSQmpqqm/EzJQpU5gxYwap\nqamoKsuXL2fixIkkJSURFxfHjBkzKCoqYty4cSQnJzN06FDfF6cLFixg4sSJ3HjjjbRv3z5gTLff\nfjtvvfWW70vbpUuXkpmZSXJyMr169eKFF14A4NlnnyUxMZGUlBRatWrFbbfdRnJyMk2bNiUlJSUq\nvrQNS3lkEekKvA8kquq3VV6bDkwH6Ny5c99ACxAYY+oumPLI4Bmlc+DA45SUHKJFi850776oXvl7\nU3/1LY8c8mGZInIxsAr4SdXGHkBVl6lqP1Xtd/nll4c6nMiwciV07QpxcZ6fK+s/3M2fUAyrM7Gj\nQ4fJDBqUzbBhZQwalG2NfSMQ0lE6ItIMT2O/UlXfDOW1osbKlTB9Opz2joA4eNDzHGCye/9D5eWt\nZO/e6b6RFiUlB9m713Md+x/XxKLly5fz61//utK2IUOG8NxzzzVQROEXspSOeFYUXgF8o6o/cXJM\nTKx41bWrp5GvqksXcHH1+k2bulJSUv06LVp0YdAg965jokewKR0TeSI5pTMEuA+4WUSyvI8xIbxe\ndDhUw7C2mrbXUaiG1RljolfIUjqq+gEgoTp/1Orc2X8Pv3P9hrtV1aJF5xp6+O5exxgTPayWTrgt\nWgTxlYe7ER/v2e6iUA2rM8ZEL2vww23yZFi2zJOzF/H8XLbM1S9swfPF7LXHH6DF8SZQBi2ON+Ha\n4w+E7gvbmTOhaVPPe2ra1PM8VMI0ysmYxsZq6TSEyZNdb+CrWbmSDtNX0OH0ee+G8xC/AoqHuH/t\nmTPht7+98Pz8+QvPn3/e3WuFaZSTMY2R9fAbq8cfv9Aoljt92rPdbcuWBbe9PsL5vkxUeOKJJ1iz\nZk3Qx61fv95XbK0+srKy+Mtf/lKnY0+cOMHzbneKArAGv7EK02ggwNOjD2Z7fYTzfRlKSnL5+ONr\nKCk52qBxqCplZWV+X3vqqacYOXJkyGM4d+6c3+3W4McyJ/lll3LQeU+PZFO6sP4fwqZ0Ie/pCv/o\nO3cmbwRs+gOsX+v5mTcC10cDAdCkSXDb66Om+EPxvgzZ2Qs5cyabgwcXunK+uXPnVmrgFixYwK9+\n9SsWL15M//79SU5O5sknn/ReO5vrrruOmTNn0qdPH77++mumTJlCYmIiSUlJvto1U6ZMISMjA4At\nW7YwePBgUlJSGDBgAEVFRZw5c4apU6eSlJREWloa69atqxbXN998w5133klycjIDBw5kx44dvvim\nT5/OqFGjuP/++6sdV1payhNPPEF6ejqpqamkp6dz6tQpHnzwQfr3709aWhp/+tOfANi1a5evjlBy\ncjJffPEFjz32GF9++SWpqak8+uijrtzjgJwWzg/HI+oXQHntNdX4eFW48IiP92wPZh8Hjv5ihG74\nK5UXqPgrevQXIzyvr3rI/+urHnLzHXs89FDl91P+eCgE13Lp/sWiYBZAUVU9cyZHN2xo6V38pJWe\nOZNb7xi2bdumN910k+/5ddddpytWrNAf/OAHWlZWpufPn9exY8fqhg0b9KuvvlIR0U2bNqmqamZm\npo4cOdJ3bPliJw888IC+8cYbWlJSot26ddPNmzerqmphYaGePXtWn3nmGZ0yZYqqqu7evVuvvvpq\nLS4urrR4yaxZs3TBggWqqrp27VpNSUlRVc/iKH369NHTp0/X+J6WL1+uDz/8sO/5vHnz9NVXX/XF\n+J3vfEdPnjyps2bN0te8/05LSkr09OnT+tVXX2nv3r0d37/6LoBiPXw3Ockvu5SDPtB9LWUtK28r\na+nZDnCg41/8v96xbh89A3r+eXjooQs9+iZNPM9D8VE1TKOcjKd3r+pJo6ied6WXn5aWxrFjx8jJ\nyWH79u20bduWHTt2sHr1atLS0ujTpw979uzhiy++AKBLly4MHDgQgO7du3PgwAFmz57N3/72N9q0\naVPp3Hv37qVjx470798fgDZt2tC0aVM++OAD7rvvPsBTZrlLly7s27ev0rEV97n55pspKCigsLAQ\ngDvuuINWrVo5fo+rV6/ml7/8JampqQwbNowzZ85w6NAhBg0axC9+8Qv+4z/+g4MHDwZ1TrdYgx+M\n2lIxTvLLTnPQtVyrpIY6c+XbHc+0dZBeclSEbcgQSEjwNMIJCZ7nJmqVlOSSl7cc1VIAVEs5enS5\nK7n8CRMmkJGRQXp6OpMmTUJVmTdvnm/Bkf379/P9738fgIsuush3XNu2bdm+fTvDhg3jueeeY9q0\naZXOq6p4KrpQbXtt/O1Tfq6KMTihqqxatcr3fg4dOsR1113HPffcwzvvvEOrVq0YPXo0//jHP4I6\nrxuswXeqfDjgwYOeREL5cMCKDaST/LKTfRxcq8Ux/6cp397ilP9/pJW2O7hOeRE2z6xd9RVhq9To\nO7k3bgnntWJYxd59Obd6+ZMmTeL1118nIyODCRMmMHr0aF5++WVOnjwJwJEjRzh2rPo/8OPHj1NW\nVsb48eNZuHAh27Ztq/R6z549ycnJYcuWLQAUFRVx7tw5brrpJlZ6/33s27ePQ4cOce2111Y6tuI+\n69evp3379tU+QdSk6iIoo0eP5je/+Y3vj8inn34KwIEDB+jevTs/+tGPuOOOO9ixY0fYF1CxBt8p\nJ6kYJ7Nonezj4FrdX21OXJVV2+LOeLYDdP/1Kf+v//pUUNcJtLZpMOdxjQ3LDIuCgnd8vftyqqUc\nP/6nep+7d+/eFBUV0alTJzp27MioUaO45557GDRoEElJSUyYMMFvI3jkyBGGDRtGamoqU6ZM4emn\nn670evPmzUlPT2f27NmkpKRwyy23cObMGWbOnMn58+dJSkri7rvv5pVXXqFFixaVjl2wYIFv0ZPH\nHnus1lWyKho+fDiff/6570vb+fPnc/bsWZKTk0lMTGT+/PkApKenk5iYSGpqKnv27OH++++nXbt2\nDBkyhMTExLB8aRuWBVCcClm1zJUrPQ3CoUOenvSiRcHnfOPiyLtZOTANSq7w9KS7vwQd/iFQcbiY\ng2vlPT2SA93XUnI5tMiH7gdG0GHemkrXwt9/F6lwrbg49s1Wcm4HmgDn4ap3ocdvvPuIkDeC6vGu\n5cK5HVxn/fo4wN+/EWHYsAux1BqvW8J5rUbGqmVGv/pWy2z8M21dmpmZd0c8e2ec8n0RWnIl7J0D\nXBxPh4o71jKLNu/NmexNu/CFa0kH2HvJWnhzJh3u8n7JedllUFBQ/eDLLqsUz9HbTl34L9gUjt4G\nlxz0xiNCh7XqaeArqpjjdHAdR0XYwlQQLuzXMqaRafwpHbdGxdxT7H/Uyz3FwZ2nbJn/85QFNyu1\n1nhq+uQW5Cc6R0XYwlQQLuzXMqaC9957j9TU1EqP7373uw0dVlAafw/fpZmZJe39pwtq2l7jeS7z\nP/u00vZvvvF/cIXtrsTj4DrlxdYCrm1a/ommvmkzJ8J5LWMqGD16NKNHj27oMOql8ffwHc44rW3o\nYYtv/M8arba9lmGOjs7jYCRPredxMvvV4azViFvbdPJkz+pgZWWen9bYG+NIo2/w8/5rDHvneHLu\nxF3Ivef914XFt5wMPeweN93/qJe46Rc2OBgy6Og8DtIW7VoOq/5dqnq3w4XvKaqaHtx1HLGhksZE\nhUbf4DuZcepk6GGHu57n2tMPVa4vf/qhC1+0gqPvCxydx8Fs0oJ2+6uvJybe7eBs9qtbs1ZtqKQx\nUaHRD8t0MqzQ0dBDJ8I4ZNC1mN0QaUMl3RiG2wjZsMzoF8mLmEeEmtZwrbjdyT6OhLGSo2sxuyGS\nKlhaeinq5OTkMGHChKCPmzZtGp9//nnAfV544QV+//vf1zW0SupbyvjZZ5/ldNVPwmHW6Bt8J8MK\nXVv/NYxDBiNqzdpIGipp6SVXbNzYhvXrpdpj40Zn5QaCcdVVV/nKG1dUU/35ci+99BK9evUKuM+M\nGTP8ljWuC2vwI0WAkTEdOkzm2muX0aJFF0Bo0aIL1167rNJIEyf7OBLGSo6uxeyGSKpgaQukuOL8\nef/1XWra7lRN9fATExMBeOWVV5g4cSK33347o0aNoqysjJkzZ9K7d2/GjRvHmDFjfH8chg0bRnkK\n+OKLL+bxxx8nJSWFgQMHkpeX5zv/M888A8D+/fsZOXIkKSkp9OnThy+//JKTJ08yYsQI+vTpQ1JS\nkq92vT/+atf7q+N/6tQpxo4dS0pKComJiaSnp7N06VJycnIYPnw4w4cPr9c9rBendZTD8ahTPXyr\nj24q6tLFf23+Ll0aOrIGF0w9/IrrKFR91Ie/evgbNmzw1YRfvny5durUSQsKClRV9Y033tDbbrtN\nz58/r7m5uXrppZfqG2+8oaqqQ4cO1S1btqiqKqDvvPOOqqo++uijunDhQlX11LNfvHixqqoOGDBA\n33zzTVVVLS4u1lOnTunZs2e1sLBQVVXz8/P1mmuu0bKyMr+xV61d/9577/mt45+RkaHTpk3z7Xfi\nxAlVVe3SpYvm5+fX5/ZZPXz7CG8qiaT0kqnGXz38zlW+67nlllu4zFve44MPPmDixInExcVx5ZVX\n1tg7bt68uW992r59+5KdnV3p9aKiIo4cOeKbGduyZUvi4+NRVX72s5+RnJzMyJEjOXLkiO/TQW1W\nr17tt45/UlISa9asYe7cuWzcuJFLLrkkmFsUUiFr8EXkZRE5JiKfheoaQHg/wru0NKEJoUhKLxm/\nqtbDr6pi/Xl1OIqwWbNmvvr1TZo0qZb/r+k8K1euJD8/n61bt5KVlUWHDh04c+aM332r0hrq+Pfo\n0YOtW7eSlJTEvHnzeOqppxydLxxC2cN/Bbg1hOf3CNcIERv9ET1sJm5Eq1oPP5AbbriBVatWUVZW\nRl5eHuvXr6/TNdu0aUNCQgJvv/02ACUlJZw+fZrCwkKuuOIKmjVrxrp16zjorzCfl7+69/7q+Ofk\n5BAfH8+9997LnDlzfHX7w1373p+QNfiq+j5QQ7EWF4XrI7yljkwMadKkdVDbg1G1Hn4g48ePJyEh\ngcTERH74wx9y/fXX1zlF8uqrr7J06VKSk5MZPHgwR48eZfLkyWRmZtKvXz9WrlxJz549azy+au36\nmur479y507dY+aJFi/h//+//ATB9+nRuu+22Bv3SNqQTr0SkK/BnVU0MsM90YDpA586d+wb6C1uj\ncEy0ibTJRcYEKVonXp08eZKLL76YgoICBgwYwIcffsiVV17Z0GE1iKivh6+qy4Bl4JlpW6eT1FKD\n3hVWh92YBjFu3DhOnDhBaWkp8+fPj9nG3g0N3uBHjUWLKi+kAjb6w5gwqGvevq4KCgoYMWJEte1r\n166lXbt2YY3FbdbgO2V12I2JCe3atSMrK6uhwwiJkDX4IvIHYBjQXkQOA0+q6v+E6nphEY7UkTEh\npKq+4YsmurjxfWvIGnxV/V6ozm2MCV7Lli0pKCigXbt21uhHGVWloKCAli1b1r5zAJbSMSZGJCQk\ncPjwYfLz8xs6FFMHLVu2JCEhoV7nsAbfmBjRrFkzunXr1tBhmAYU/bV0jDHGOGINvjHGxAhr8I0x\nJkZE1Jq2IpIP1KG2gqvaA8cbOIZgRVvMFm9oRVu8EH0xR1K8XVT1cic7RlSDHwlEJNNpXYpIEW0x\nW7yhFW3xQvTFHG3xlrOUjjHGxAhr8I0xJkZYg1/dsoYOoA6iLWaLN7SiLV6IvpijLV7AcvjGGBMz\nrIdvjDExIqYbfBFpIiKfisif/bw2RUTyRSTL+5jWEDFWiCdbRHZ6Y8n087qIyFIR2S8iO0SkT0PE\nWSWm2mIeJiKFFe7xEw0RZ4V4LhWRDBHZIyK7RWRQldcj6h47iDfS7u+1FWLJEpFvReQnVfaJmHvs\nMN6Iuse1ifVaOj8GdgNtang9XVVnhTGe2gxX1ZrG/t4GfMf7uB74rfdnQwsUM8BGVR0XtmgC+zXw\nN1WdICLNgSqLJUfcPa4tXoig+6uqe4FU8HS2gCPAW1V2i5h77DBeiKB7XJuY7eGLSAIwFnipoWNx\nyb8Av1ePj4FLRSTwCtHGR0TaADcB/wOgqqWqeqLKbhFzjx3GG8lGAF+qatWJlhFzj6uoKd6oErMN\nPvAs8FMg0Ark470fKzNE5OowxVUTBVaLyFbvwu9VdQK+rvD8sHdbQ6otZoBBIrJdRP4qIr3DGVwV\n3YF8YLk3zfeSiFxUZZ9IusdO4oXIub9VTQL+4Gd7JN3jimqKFyL3HlcTkw2+iIwDjqnq1gC7vQt0\nVdVkYA2wIizB1WyIqvbB85H3YRG5qcrr/la0aOghWLXFvA3PtPAU4DfA2+EOsIKmQB/gt6qaBpwC\nHquyTyTdYyfxRtL99fGmn+4A3vD3sp9tDfrvuJZ4I/Ie1yQmG3xgCHCHiGQDrwM3i8hrFXdQ1QJV\nLfE+fRHoG94QK1PVHO/PY3jyiAOq7HIYqPgpJAHICU90/tUWs6p+q6onvb//BWgmIu3DHqjHYeCw\nqn7ifZ6Bp0Gtuk+k3ONa442w+1vRbcA2Vc3z81ok3eNyNcYbwffYr5hs8FV1nqomqGpXPB/V/qGq\n91bcp0re8A48X+42CBG5SERal/8OjAI+q7LbO8D93lEOA4FCVc0Nc6g+TmIWkStFPGvticgAPP8e\nC8IdK4CqHgW+FpFrvZtGAJ9X2S1i7rGTeCPp/lbxPWpOj0TMPa6gxngj+B77FeujdCoRkaeATFV9\nB/iRiNwBnAO+AaY0YGgdgLe8/66aAv+rqn8TkRkAqvoC8BdgDLAfOA1MbaBYyzmJeQLwkIicA4qB\nSdqwMwFnAyu9H+EPAFMj/B7XFm+k3V9EJB64BfhhhW0Re48dxBtx9zgQm2lrjDExIiZTOsYYE4us\nwTfGmBhhDb4xxsQIa/CNMSZGWINvjDExwhp8Y7y8lQ/9VU71u92F690pIr0qPF8vIlG3TqqJHtbg\nG9Nw7gR61bqXMS6xBt9EDe/s3f/zFqr6TETu9m7vKyIbvEXa3iufJe3tMT8rIh959x/g3T7Au+1T\n789rA13XTwwvi8gW7/H/4t0+RUTeFJG/icgXIvKfFY75vojs88bzooj8t4gMxjODe7F46qhf4919\noohs9u5/o0u3zhjAZtqa6HIrkKOqYwFE5BIRaYanaNW/qGq+94/AIuBB7zEXqepgb+G2l4FEYA9w\nk6qeE5GRwC+A8Q5jeBxPKY4HReRSYLOIrPG+lgqkASXAXhH5DXAemI+nzk0R8A9gu6p+JCLvAH9W\n1Qzv+wFoqqoDRGQM8CQwsi43yhh/rME30WQn8IyI/AeehnKjiCTiacT/7m0wmwAVa6/8AUBV3xeR\nNt5GujWwQkS+g6cSY7MgYhiFp/DeHO/zlkBn7+9rVbUQQEQ+B7oA7YENqvqNd/sbQI8A53/T+3Mr\n0DWIuIyplTX4Jmqo6j4R6Yun1srTIrIaTxXOXao6qKbD/DxfCKxT1e+KSFdgfRBhCDDeuxrShY0i\n1+Pp2Zc7j+f/L3/lfgMpP0f58ca4xnL4JmqIyFXAaVV9DXgGT5pkL3C5eNdzFZFmUnkRivI8/w14\nKi8WApfgWa4Ogi+K9x4wu0KFxLRa9t8MDBWRtiLSlMqpoyI8nzaMCQvrQZhokoTnS84y4CzwkKqW\nisgEYKmIXILn3/SzwC7vMf8UkY/wrFtcntf/TzwpnUfw5NSDsdB7/h3eRj8bqHE9U1U9IiK/AD7B\nU9f9c6DQ+/LrwIsi8iM8VReNCSmrlmkaLRFZD8xR1cwGjuNiVT3p7eG/Bbysqv4WwzYmpCylY0zo\nLRCRLDwLwHxFhC+DZxov6+EbY0yMsB6+McbECGvwjTEmRliDb4wxMcIafGOMiRHW4BtjTIywBt8Y\nY2LE/wcLCmK1wPtUPgAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0xc11bd30>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "# 训练集\n",
    "plt.scatter(X_setosa_train[:, 0], X_setosa_train[:, 2], color='red', marker='o', label='setosa_train')\n",
    "plt.scatter(X_versicolor_train[:, 0], X_versicolor_train[:, 2], color='blue', marker='^', label='versicolor_train')\n",
    "plt.scatter(X_virginica_train[:, 0], X_virginica_train[:, 2], color='green', marker='s', label='virginica_train')\n",
    "# 测试集\n",
    "plt.scatter(X_setosa_test[:, 0], X_setosa_test[:, 2], color='y', marker='o', label='setosa_test')\n",
    "plt.scatter(X_versicolor_test[:, 0], X_versicolor_test[:, 2], color='y', marker='^', label='versicolor_test')\n",
    "plt.scatter(X_virginica_test[:, 0], X_virginica_test[:, 2], color='y', marker='s', label='virginica_test')\n",
    "\n",
    "plt.xlabel('sepal length')\n",
    "plt.ylabel('petal length')\n",
    "plt.legend(loc = 4)\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 6. KNN算法总结 "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "KNN算法是一种最简单最直观的分类算法。它的训练过程保留了所有样本的所有特征，把所有信息都记下来，没有经过处理和提取。而其它机器学习算法包括神经网络则是在训练过程中提取最重要、最有代表性的特征。在这一点上，KNN算法还非常不够“智能”。但是，KNN算法作为机器学习的第一个算法，还是值得我们了解一下的。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.3"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}
